Getting into Terraform, Part 4: Internals

Hello everyone and welcome to getting into Terraform part four. Yes, we were a little late this morning. That was my fault. I'll take the blame for it. Uh welcome to part four where we are going to be discussing internals. Um I You know, I should just start out with the introduction. I'm Rosemary, this is Richard. We are here to learn Terraform in the most inefficient detailed way possible. So, if you're looking for a Terraform 101, if you're looking to get a Terraform certification, I'd highly recommend another course. Uh while we do go through all of the aspects of Terraform and how to use Terraform, we are also not going to be the most concise. Uh yeah, that's just the way this goes. If you've never tuned into a getting into series before, welcome. Um we learn everything in the most inefficient detailed way possible. So, you will see us take tangents. uh sometimes be a little long-winded in explanation and being a little confused about things. Um we're not going to show the happy path. We show the sad path, the path of basically everything not working. Um and so, here we are. Uh we are going to um continue the series throughout the year because this is going to be a very long one. So, if you missed any of the previous episodes, uh you can get um check out our playlist. We've had three episodes so far. Um and in keeping with the series, we love for you to interact in chat. I see some folks in chat now who are saying hello. We want you to give us comments. your uh hot takes, some tips and tricks as we struggle along. Um so, if you do participate in chat though, uh please follow our community guidelines. Be welcoming, be respectful, and professional. Um we have our community guidelines online, so definitely check them out. Make sure that you uh as you interact with us, um you're respectful of folks other folks who are in the chat as well. Um am I missing anything in that instruction? I don't think so. I think I got everything. No, I think you got it. All right, Richard. What the past three episodes, I feel like we did everything and nothing. Yeah, we uh it's so much uh trying to distill it in a few words. Uh yeah, so we went over it very briefly uh state uh how state's managed, where it's managed, uh things to keep in mind when you're mutating state if you're working on a team, how to keep state from getting corrupted uh was a huge part of the first episode. And we talked uh very briefly about uh providers and like the actual HCL the HashiCorp configuration language. Um I have a question about that for you that I picked up this last week that I want to ask you about once we get to it. Uh and we covered like you know, variables, uh resources, outputs, like the key parts of a Terraform template. And then we went into like more specifics of the HCL, which is uh meta arguments. We went over when you would use count or for each or historically how those came about. Uh we talked about functions, expressions, uh and I think that was and that's the most of what I can remember. Yeah, so we were I would say we went really deep in sort of playing around with the configuration language and debugging and things. Uh I guess in the past three episodes, what's the one thing that you've learned that's changed your Terraform development workflow? Uh I think for you like the for each and count like I come from a more CloudFormation-esque background where it's like you need 10 of a thing, copy paste like that's why keyboard has those keys. Uh I think you like I used that for the first time the other day when I was uh creating a DKIM entries for like sending a Amazon SES email or receiving SES emails cuz there's like a thing you have to iterate over that I had historically just copied and pasted. You know what's worse than copy and paste? Copy and paste it but also to make one little change to each of them. I'm like, oh, that's silly. Uh, you did you've learned the power for each. But with every iterate like with every loop, with every iteration uh, capability, there's also some pitfalls, right? Well, you know, as you noticed before like you have to be really conscientious about how you iterate, what values you're passing in, right? There's logic that's not as clear, right? And there's a point in which you would copy paste too. That is kind of cool. Anyway, hello everyone. Hello, hello. Welcome in. If you have questions, please post them in the chat as well as we go along. Um, you know, for if you want us to pause and talk a little bit more about something we can. All right. So, let's talk about internals today. So, we are not going to clone Terraform, fork it, and make edits. Um, I just I think the compilation of Terraform is fine.

You can do it on your own machine. It is source available. But by the time we try to fix Richard's machine to clone and like actually compile it, I just don't think it's worth it. But what we will do is we're going to dive a little bit into each of like what this what the internals are and then maybe relate it to some of the code bits and kind of do a little bit of talking. Um, and you may see us do some editing, doing some Terraform stuff related stuff. So, Richard, you'll probably need to pull up two IDEs, one for the uh, code base we've been working on, the demo code. I'll put up the uh, hold on. Where's my mouse? demo code repo so folks know. Uh, we do have a demo code repo. Um, and so those we're going to have that available and we're going to make some edits to that to just test out what this looks like from an end user perspective, but then we're going to dive into the internals. So, uh Richard will also have a local uh he has it locally um basically to take the Terraform repository uh for reference. Um if you do want to go really deep into Terraform, um big shout-out to Daniel. Uh Daniel is one of the folks who works on Terraform, and fantastic engineer. Um and if you want to learn more about the deep internals of Terraform, uh he does have a series where he talks about them. Uh so, if you are planning to contribute to Terraform, or you're just curious about sort of what it looks like to contribute to Terraform, uh and all of the little nuances that are available to within Terraform itself, um his blog post will go into exactly all those pieces. Um so, he talks about the actual deeply into the go code base as well. So, I'll post that up. Shout-out to him for uh doing that. Yeah, I like Daniel's content. I used his guide on Terraform actions to make the provider actions demo I created. Yeah, he's great. All right. Now that I'm sharing my whole screen, I have moved the window with us talking over here. So, if you can see me looking over here, I'm not watching the highlights of yesterday's game, I'm actually looking at you. — Why do you have a Giants cap? Uh I like to switch it up. Uh and the Red Sox are currently last in the MLB, so I'm not wearing my Red Sox cap. — Uh okay, all right. For those who are not in the United States, uh baseball team. That's why I was confused because I was I thought Richard was a Red Sox fan. I am was like, why do you have a Giants cap? All right. Okay, so let's get started. Um I'm going to pull up what my list of things. Okay, so the first important thing that we're going to talk about is back end. And the reason why we want to talk about back end is up to this point, Richard, we've been using our demo repository against a local repository. So I'm going to pull that up. You've got both pulled up. If you can make the getting into Terraform repository bigger, like full screen. Yeah. Oh, yeah. Yeah, we'll do it that way. Yeah. And then just zoom in cuz it got wide screen. Keep zooming. Okay. Okay, so the first thing we're going to do is we did a little bit we did talk a little bit about back ends and about state, right? But what up to this point we've been doing local state. I don't know if we can pull up the main. tf. I think it's in there somewhere. main. tf might have it defined or maybe not. No. So by default you can define state to a local state file, right? That's that terraform. tfstate. But the reality is this is probably not the greatest thing to do for a couple reasons. The first is that it's local, which means not if Richard wanted me to also create resources in his AWS account, I would not know that these resources already exist or Terraform doesn't and thus, you know, now I create a new state then potentially override whatever Richard's done. So it's not so great. When you collaborate, you do want to use a remote back end. And that way it is it's something so you can persist state data and that way someone else who wants to make changes to Terraform has the ability to at least know that these things exist. Now, we can we have a couple different back ends. I'm going to put the documentation in there. We have a couple different back ends. We have S3 buckets, we have all sorts of things. I guess I'm going to ask Richard, do you want to do a remote back end on HTTP Terraform? Do you want to do it on I don't know, S3? What do you feel like doing? Uh we can do either one. Which would you prefer? Sorry, I was just putting the link in the LinkedIn. Sorry, I've got it back. Yeah, oh thank you. Yes. Yeah. I have been doing a lot with HTTP Terraform and S3.

Uh okay. Over the past few days, so Do we even have an S3 bucket available for us to write to or not? — I've got so many S3 buckets. Okay. Why don't we do it to S3? I mean, we could do it to HTTP Terraform, it's the same thing, but you know, we'll show it in S3. Actually, let me just S3 make bucket Yeah. Richard and Rosemary Yes. and Terraform and then something to make sure it's unique. Yeah. Yeah, yeah. Okay. Uh Oh, chat has spoken. We're going to do S3. Okay. Here we go. Uh and why are we doing this? We're doing this first because this is inevitable. You do want to by extension, you should definitely put them in a remote state somewhere. Uh this stops with the collisions and all sorts of things and ensures that you could preserve your state somewhere. Uh so, Richard is just going to create an S3 bucket. Um Hopefully. Uh why is this not I don't I thought it was just make bucket, right? Uh I think it's complaining that S3 is not it doesn't like S3. Why would it I mean, that's like a real thing, right? I don't remember. I've never created it with the CLI. The last time I created anything with anything in AWS with the CLI was probably 6 years ago. I don't know ask Bob. ask Bob. — Oh, yeah, yeah. Let's do that. Let's So for those who are not familiar with what's going on, this is a high, this is Bob. Bob is a coding agent. Um You know, you could tell Bob to do it for you, you know. Uh Bob is a coding agent IDE, so there's a coding agent in it and you know, you can use it to code stuff. Yeah. I mean, there are many things. There are many things you can do with Bob. One of these — This is exactly what I uh Maybe you see a lie is not installed. Did you define the region or did you unset it? — It it is. Uh uh I promise my CLI set up correctly. I at least I Are we deploying it to the same region? Are we deploying everything Yes. Uh Let me try. Wait, I thought no, bucket names are global though, right? So Yeah, I think all right. I think what I did was the first time I said like Richard and Rosemary and blah blah, I think the bucket name was too long, so it complained. But it just says like that's wrong. It doesn't say why. So then I was like, okay, let me make it shorter and I just gave it like a bunch of numbers. I don't think you can start a bucket with a number. No, I guess not. — complained about that. So I hit two different errors, but it gave me the same error message, so I thought, okay. Okay, so we now have a bucket. Yeah, we have a bucket. All right. Okay. Yay! All right, so for S3, we do uh it's in the Terraform stanza. Uh so you do Terraform. And then, yeah. And then back end double quote S3 space double quote S3. Mhm. Oh. Uh yeah. And then curly bracket. Yeah. And then bucket equals the name bucket. Yeah, we could use an existing bucket with a unique path key, but I think actually Richard's buckets are for other things. So, we probably don't want to mess with them. I don't know if you have a spare bucket a bucket to spare for us. Okay, and then key hit enter key. So, key is key is going to be whatever your path to your state file. So, if you have a key, you could do that or you could just not define a key. So, does that include the state files name itself? No, no, no. It's just the path. Just the path portion, okay. Yeah. Uh you just leave it blank. Yeah, you could leave it blank or you could just not define it. Oh, yeah, okay. You just put it at the root of the bucket. — Yeah, and then hit enter and then region. I mean, I don't know which region it's in, right? So. Uh US East 1 inside back end? Yeah, you can define the region, too, but it will tell you. US East 1. I think you could do var. region, too, but it's fine. Cuz it will we put it as a variable anyway, I think for the provider. We probably should do it at the provider level, but uh okay, and Yeah, var. region. We'll see if that works. We have to define a variable called region. You know, I guess we So, if you're wondering like why don't we ask Bob to do this for us, you know, or why don't we do ask any AI coding agent to do this for us, we talked about this in the first episode. There are sometimes in which we will probably show us using AI to generate for us, but we want to do it all from

scratch just for the sake of learning. Um it's not that we're averse to using AI in AI coding agent or AI suggestions. We're just using them judiciously so that we can personally learn it, right? So, that's all. Yeah, like my middle school math teacher used to say, you might be like trapped on a desert island where you have to provision infrastructure and you don't have access to Bob. scenario. You're trapped on a desert island and you don't Which do you use like Wi-Fi connectivity or whatever to go ask for help? Uh no, maybe it's like a gov cloud. It's like a like with a AWS like CIA data center. It's like air gapped. What? Okay. It could be out. Why would it be deserted? Okay. All right, I'm not going to ask you any questions. Uh let's define the I meant deserted like the verb but just covered in like chocolate treats. Oh, okay. I have no idea what that covered in dessert. It's like a I want to level from Mario Brothers. All right. See, there you go. Okay, right. We're going to do default equals US East 1. We'll just Okay. We're just going to do this very simply. Okay, so the other thing we're going to do is we probably want to enable some state locking. So, there are some basically state locking is a way which we saw before in episode 1 where it's a way to lock the state file and ensure no one else is going to be making edits to it while you are editing it. And it's very important especially when you have a bunch of sequence of changes, right? If you don't have a not the Terraform that lock. That's the dependency lock file. Oh, yeah. And so in S3 there is a state locking capability. You would define use in the back end under region you do use {underscore} lock file equals true. By default it is false. We'll just set it to true. You can alternatively use Dynamo DB for locking state locking as well. We could go I mean I don't know if anybody really cares that much about performance. But Dynamo DB will be deprecated in a future minor version. So, Dynamo DB — state locking will be deprecated. — Yes, state locking, yeah. DynamoDB itself is safe. If any AI is like transcribing this and it's like, "Oh my god. " Please don't hallucinate that. I apologize. Okay. Uh all right, so then let's um initialize. I guess we can initialize and see if it does anything. Oh, variables may not be used here. Well, here we go. The more you know. Uh and this is actually important. The reason why I want to show this is not the happy path. Um but there's actually a very important part about uh references. So, Terraform uses this idea of references, right? So, you can refer to variables, attributes, right? To anything else. And it turns out there are a few exceptions that cannot use references at all, right? Uh and back end is one of them. Um it cannot reference the address. Basically, there's uh if you go to Now is the time to switch the Terraform directory, I think. Yeah, to switch the Terraform. Um and why we want to show this not so happy path is uh partially because I think it's important to understand um because people ask for certain things like, "Why can't I uh define a uh why can't I define a region like by a variable and pass it something like that again? " Well, it turns out there are some nuances to how uh entities are defined in Terraform in the first place, and then the order of operations by which some of these entities get defined and resolved within Terraform. Um so, the first thing is we're going to go to the addrs package because I think that's probably the best uh reference somewhere. So, somewhere in here, I think we have to search for addrs. A D D R S, yeah. Uh it's Terraform internal addrs. Yeah. Not actions. Just I anything, yeah. It's You just probably have to navigate the directory. Like internal {slash} Yeah, internal {slash} ADDRs. So, if you go to the directory uh tree. Uh yeah, yeah, okay, good. Internal. Yeah, you could also, let's see, ADDRs. I was trying to be clever. I thought that readme was in a different spot and I could like go and be like open in I was trying to be more clever than I am. It's okay. Yeah, so these are actually the different entities. Um I think that should be There should be one for resources. So, if you scroll up and down, there should be one for resources. Um there should be one Yeah, see resource. That's the most common one, so you can look at that. You can also remove the terminal. Hit click X on the terminal, we get more We're more space. Uh so, these are sort of references, right? So, the ADDRs, anytime you have a

reference within Terraform, this is the package that's uh that's it's using under the hood to reference something, right? Um and so, resource is one such capability. If you scroll down, there's like you know, count for each is also in itself its own implementation, but if you keep scrolling down in this directory, um there's resource, right? So, it has all sorts of uh functions and things like that. Like you can eat keys. Um instance defines like that instance key. So, that's that identifier. So, you'll see like AWS the AWS un underscore instance and then you'll have that identifier. That's the instance. Um and so, then there's the absolute address of the module, which is import resource, which is very important because maybe you'll define a relative uh you'll define it relative to a module, but this one is the absolute. So, like module. aws_instance, whatever. Uh and then it also some definitions of what it is in the module, what the provider might be. So, this is all metadata um sort of about the address or the entity of a resource. Yeah, so there's a lot going on in here. Uh is there one for variable? That is one thing that I wonder. Is a variable not in here uh in the file a variable. go in the file structure. I wonder if that's Terraform attribute is also See, maybe this one. Uh yeah, so this is Terraform attribute. This is like if you were to interpolate like terraform. workspace or terraform. something, there's even an attribute for that. I don't know if variables is in here. Let me I have to scroll down down. Uh Yeah, I don't think variables is defined in addresses, but effectively this is when you're trying to resolve information about resources and things like that, this is where you're going to want to look. There's a lot of entities um that you could look at, but most of it is um pretty much related to resources and I think modules. All right, so let's go to back end the back end directory If you go to back end, yeah. So, uh there should be remote or pluggable or one of them. I think S3 might be in a separate directory entirely and separate. Um oh, there we are. S3. Okay. Back end. go I think is the one. Yeah, so there's a lot stuff going here. Um region is not in here surprisingly. Does that mean we don't need region? I mean, I guess it's implicit, but I don't think that's the case. Well, uh because I assume it's because the bucket names are globally unique, you wouldn't have to specify the region. Like if you give it a bucket name that implies a specific region, which if it's using S3 client, it's probably doing like list buckets, finding out what region the bucket's in, and then going to that, setting that somehow. That's true. I don't know. We can keep looking at this cuz I think there's probably something in here. It does reference It must reference because it's an S3 client though, right? Because S3 client initializes with a region. Cuz the client initializes with — but I think you can I know nothing knowing nothing about the Terraform internals, you could initialize with any region, almost any region, and then say list buckets, get a bucket that has or just like Yeah, yeah. See what the name of that bucket and get the information for that bucket, see what region that bucket is in to then go like update your region to point to that. My question is like, does this mean I could do a remote back end and say US East 2 forward thing I'm deploying in US East 1? Maybe. I mean, we could try it. I would just So, first step, let's click on that S3 back in S3 client, like search for references. Not to the pointer, the one to the left side. So, and go Yeah. Uh upper a core um Do we have to ask Bob for this? I don't want to do it. I There should be a way to right click on it, finds all the instances that it references. Don't tell me when I did this. Oh. Okay. I don't know I don't have the extension. — this happens, while this happens, I will review chat for a second. Uh Yeah, Chris is asking if um Bob or Terraform will help him plant orange trees uh in zone five. And I would say you can plant them wherever you want. It's just whether or not they'll grow. Yeah. I think you need a greenhouse. And yes, this is Bob. We are showing Bob. We are using it uh judiciously. We're not going to go running around using it for everything, but you know, we'll use it for some things. Uh Go Yeah, go back to back-end. Yeah, okay. So, click on that.

— Oh, so many more extensions. — Ah, yes, yes, yes, yes. Okay. All right, go to references. Find all references. Uh oh, yeah, well, that also works, too. Um yes, so — definition, no. No, it's — No, no, no. Go to find you could go right click and say go to references. Uh fourth one. Fourth not find all. Go to the fourth one. Go to references. Uh is it thinking? Oh, boy. You know what? You might as well control F S control F on the S3 client. Do it. We'll do it the manual way. By the time it indexes everything and I don't have it in me to wait. Uh okay, yeah, there you go. So, list buckets. Okay, so that's act that's testing. I don't want the tests. Uh all the tests might sometimes give you something you know, something good to know. Um but not the tests. We want the actual like uh reference and back-end. go. So, new from config. So, okay, region. This is a Okay, so it's not actually adding region anywhere cuz it's new from config. So, it's using AWS That's just important. So, it's using your AWS config. So, that's probably why region isn't necessary even. Cuz look, it's using it from AWS config, which could be by SDK or could be defined by config. So, I wonder if we remove region. It's like not necessary. I mean, we could still do it, but like I don't see it. — Let's test it. Let's try this. Okay. — Make bucket. Oh, crap. Uh let me try that again. It's going to tell you to delete it. Well, I'm going to make a new one. — Oh, you're going to be globally unique. Okay. For those who are wondering, Chris our manager is in chat. We're now just messing around. All right. Okay. So, it's in US East 2. For those who are following along, this is in US East 2. This bucket, Chris, Uh and so, we are going to see Uh oh, we have to do a path. We do have to define a key. All right. Well, hit control C and then we'll add a key. Okay. Yeah, I guess we do have to define it. It's fine. Region is an optional, but key is not. So. Uh You Yeah, you don't need the backslash. You can just do the relative. Oh, just empty? Oh, yeah. Okay. Or maybe you do have to do like the file. I don't know. I don't remember. Oh, you have to be uh Fine. I don't think that will work either. Ah. YOU HAVE TO DO IT. JUST SAY — KNOW. I THINK it's No, no. I think I have to name I put the name of the state file there. Well, you Okay. — key I think they're using key in the context of S3 key that it's like the file plus all of its path. Oh. Great. Okay. — my state file. I'm sure there's like a Terraform idiomatic one. Yeah. No, I mean terraform. tfstate, but You know, like it's been a Like I said, it's been a while since I've done any of this like in a CLI or even done S3 initialization. Must be set. Okay, so yeah, so it is picking. So the region attribute is there. Okay, so then we knew we went on a tangent. Basically, I was explaining why this has to be hardcoded and then we went on this tangent. It's okay, we'll come back to it. Uh okay, so US East 2. This is fine cuz we know it's in US East 2, but what if you put the region in US East 1? Right? Cuz then it doesn't exist. That's fine. You can migrate state. I think it will error out. Yeah, it doesn't exist. Yeah, so it does happen. Well, because it's trying to yeah, requested bucket from actual location. Yeah, mhm. Okay. Yeah, so it doesn't work that way. Um but back to our original discussion, which is that back end is one of the first thing that's initialized. It's actually initialized before variable. And so the result is that you cannot use a variable defined in back end, which is, you know, it's a little bit irritating for folks who want to, you know, uniformly define all of their

region in a variable and then pass it to back end, but you cannot use a variable in back end. You can't use any interpolation in back end for what any reference that I can recall in back end just because back end is the first initialized component. So, fun fact. Um I don't know if I want to dig through the code to show that, but yes, it does exist if you dig through there. Um okay. I think it's under evaluation. I want to say it's an evaluation, but I'm not going to go deeply into it. We could look for it. Oh no, it's not. Well, oh look, there it is. Object I get attribute region. So, it's lovely that they like it's an optional regional attribute. We found region. There it is. Uh so, that with that we know. Um I think you has to be under Terraform back end. Like it's under the back end object, I think. We can't or in knit is Yeah, it's probably not in there. It's okay. All right. So, now that we got the back end, we can now go to S3. I don't know if you want to dump out the S3 bucket and just show that the wall exists. I guess cuz we didn't create anything yet. I think because we didn't create any resources yet, the state file doesn't exist. So, we probably want to create some kind of resource somewhere. And not in US East one. I mean, I guess we could. Cross region is okay. Yeah, I mean, cross region's okay. Cuz that's like theoretically, that's how if you're using like HCP Terraform Yeah. you could see that as being something like two separate regions. Like your state doesn't have to be stored within like the same Yeah. — place and yeah. Yeah, exactly. All right, let's uh Create an AWS instance. Oh, I forgot we made these like uh Where is that instance? There it is. — All right. So, as this is creating Why don't we re-initialize it? Let's do a straight apply. Yeah. All right. While this is applying, oh, yep, there we go. Da da da da. There we go. Well, we There we are. All right. So, it's a file, same as local file TF state. Not any I would say like it's the same schema, same everything. The biggest difference is that it's now in S3, and eventually like if you if let's say I wanted to run Terraform apply against it, I would reference this S3 bucket, right? Um, and then it would lock it and make sure that this it would sequence the changes on our behalf. How does it do the locking? I assume it puts some kind of metadata tag on it? Yeah. So, we could try locking it now. I think you could do Terraform state lock or something. Terraform It's called state lock or Terraform lock. I don't remember or something like that. What was the command? Hold on. CLI. We did this before and then of course I forgot again. Mhm.

S3 How do I lock it? Enable state locking. State locking. Use files. I want to just I mean you could technically run plan and just leave it open. And then you'll be able to open a separate terminal. So we can't really lock it remotely, I don't think. Uh if you ask Bob it does the DynamoDB lock. Yeah, now it that probably not a good idea. All right. Oh, manual lock. Terraform force Oh, force unlock. — That's force unlock. We could just do a Terraform plan and just like not do just do a Terraform plan. Yeah, that's the easiest way to do this. We'll correct Bob. Just do a Terraform plan and then open hit enter. Oh. Well, you got to get into 04 internals. Oh. Go into 04 internals. Yeah, do a Terraform plan and don't do anything. Or sorry, Terraform apply — Yeah, I was confused. Yes. — Uh is this going to just Am I just — Well, I know I might not do it. Let me make a small change. Uh user data Because there's like no change, right? So like I assume it's not going to hold a lock if there's no change on the apply. — Probably not. Yeah, okay. So then now you can look at the S3 LS. So, if you There you go. Oh, it play Oh, it creates another like a sentinel file. Right. — I thought it was using like it'd be using like the — No. AWS S3 like metadata tags. No, but I believe it also probably uses a combination of the metadata tags. So, it's not It will just do basically what it's doing in uh on the local machine. Yeah, on your local machine. So, I'll just create TF lock. Uh let me have a couple hands questions in chat. Have folks observed under heavy load, high parallelization, releasing the lock locks up and never releases the lock? Has it been occurring for other folks, too? I would assume yes, that this does happen. But, I get I guess it also depends on your back end, too, and when your back end is like how responsive is your back end in releasing the lock, as well. So. Yeah. I ask you about that with S3 back end, you could just delete the file. Could what it let you do? If it's queued a bunch of requests to like create and lock the file? Could you? I mean, I wouldn't Here's the thing. If you're doing this in S3, folks, don't just give everybody access to S3. Like, put it in a separate account or something your state buckets for Terraform in a separate account or something. Oh, S3 back end in Atlantis. Yeah, I don't know with Atlantis. I've not encountered it, but uh I assume that makes sense with Atlantis. And we have not covered Atlantis yet. Um So, yeah, we haven't talked about Atlantis, which is a way to basically do PR based uh applies of Terraform. Oh, yeah, I guess it does work. Yeah. So, this one is still like waiting for me to respond. I deleted the lock file and then opened a new terminal. Yeah. Uh-uh. Okay. Yeah, it does some stuff. So, if your deploy is ever stuck, just delete the lock and push it anyways. Manually delete the lock file. Uh, I don't know how I feel about this. Oh, there we go. Yeah, cuz it is still locked cuz it's deleting. Yeah. Oh, and then this one tried to release the lock while the other one was applying. — Yeah. So, there you go. Uh, it's tomorrow Richard's problem. It's fine. I mean, it's destroying it. So, as long as it's destroying, it's destroying it. Yeah, and like when I think when this one's done, it'll release the lock and the lock will be gone. It's just Yeah. Done. See? Yeah. Problems solve themselves. Uh, I'm trying to think if there's anything else that needs to be shown with back end. Um, we could go to back it. If you go back to Terraform internals again, delete the lock file manually. What could go wrong? I know, right? Uh, so we're going to go into internal/backend/backend_base. Okay. Uh, we'll back in base. And then base. go is fine. Uh, so important thing you'll notice at the num at the top of a number of files in Terraform is this go-cty library. Um, this go-cty library is pretty critical. It is responsible for um, the basically mapping the values of, you know, whatever values in Terraform um, to some type-safe way. So, it's primarily responsible for evaluating expressions

and things like that coming uh, you know, whatever expressions that you say you want to represent. So, that could be like maps. It could be anything else. Um, but a lot of the back end, for example, a lot not just back end, but a lot of the files you'll see have this go-cty because it's using it to evaluate the expressions that are in HCL first and then gets passed through. So, part of let's say the back end schema is that it will prepare the config. So, if you scroll down, there's a function called prepare config. And prepare config It's partial implementation. It will depend on the back end schema as well, but prepare config is primarily where everything started sort of starts where it will prepare the back end schema. You'll notice that it looks for any null objects, you know, keeps going down. It will look at the schema and this is where it sort of evaluates the back end code and says, okay, are the value are the attributes that you need available in the back end code. So, that's why you'll find a lot of this is referencing cty go cty and that's because go cty is sort of like that interface or abstraction for all of these type safe components. The other important thing to notice is TF diags diagnostics. It's a library. Yeah, so these are if you've ever wondered what are the errors that are coming out of Terraform, they're called diagnostics. This is not I would say this is not something that we commonly we just say it's like it's an error from Terraform or like these are plans applies. These are all the you know, all the things that come out of Terraform, but they're technically called diagnostics in the core of Terraform's code. So, it's actually um Yeah, there you go. See, you now have definitions and they're very well commented. So, it explains a little bit of the mentality of what diagnostics are doing and it's not just producing the uh plan output, but it's also giving you information about why the plan failed and things like that. So, these are all considered diagnostics. Okay. There you go. Um there is a way to do like TF log, which is you I think we could show that. Um TF {underscore} log, I Uh not No, sorry. Go to your um Terraform Go to your Go to the getting into Terraform. Getting into Terraform repository, and then uh before apply, you would do capital TF {underscore} capital L O G uh equals debug. Uh capital debug. I think it debug. Or you could do trace, but we'll do debug cuz trace gets really noisy. So, debug capital D E B U G, all caps. All caps? — matter. I don't know. I think it shouldn't matter, but I don't know. Uh yeah, debug, and then do Terraform apply. So, what this will do is it not only uh I mean, apply the Terraform plan can also be quite noisy, um but this will basically print out pretty much every single debug statement in Terraform that Terraform does. Um inclu- Yeah, whatever. Maybe um put the make the terminal uh go um full screen. Yeah, there we go. And then maybe take the side panel, the explorer panel maybe. Yeah. All right. So, you'll notice this is pretty much every single call that Terraform makes. Uh including but not limited to the initialization step, um as well as the caller identity. So, when he when Richard makes the initial call um to create the resource, it also does a state read to find the exact current state. If you ever run into any issues with Terraform, Terraform not the plan specifically, but the Terraform itself, whether you're debugging a provider or who knows what, you can use TF log to basically dump out all the information you need. I like this. Yeah. It's invalid, but we're tolerating it. Yes. This also provides warnings from the provider side as well. So, it surfaces every single log. And so, if you ever have any issues, you can do TF log. Yeah. So, it also I mean, you could also have debugged like, you know, which region and things like that. I guess there's a big question of like, do you need this dumped out to everything? No, I've never really I've had anybody who told me that from an audit perspective, they wanted all these logs.

logs. Uh I mean, usually a TF a plan and apply is what most folks want. I've never had anybody tell me from an audit perspective, they wanted to take all these logs. Uh but maybe it's, you know, useful if you wanted to trace everything that Terraform was doing. Not sure. Does it even send debug statements? — Yeah, I feel like that's useful for like, if you're building the Terraform like a Yeah. like a provider or like building Terraform itself or the AWS provider, you would want that debug. Uh just cuz it's easier than like trying to do a bunch of print statements and then capturing that those that output, yeah. Yeah, exactly. Now, you notice these are pretty noisy. So, if you're working, let's say you're contributing to Terraform and you want to know what's going on with just the Terraform code you're working on, you could do TF_log_core. Or you could do info. And you would clean it up. It's okay. TF_log_core. underscore core That did nothing. It's fine. We'll try it again. TF log _core equals and then you can say info or debug. I don't It doesn't matter. Um what this will do is only print out the calls made by the Terraform binary client. So, this is the Terraform binary. It ignores anything to do with providers. anything else. It just does the core engine. So, you'll notice that important thing to see is that initializes the back end. Um and that's a core function. That's not part of the provider function. Back ends are part of the core function. So, if you keep scrolling down, it will tell you like, you know, some registry uh registry-related things, um remote state. Yeah, so it's a little less noisy. Now, if you do TF_log_provider instead of TF_core, this is good if you're, let's say, debugging your provider you wrote, uh then you would only get the stuff coming from a provider. So, let's say, if you scroll up, uh it's saying like automatic mTLS and then uh you know, various things here. So, these are all related to the AWS provider because we're using AWS. But, these are sort of info messages about the Terraform AWS provider specifically. So, the result is if you are trying to debug something maybe because you made a change to Terraform core or something else, you have maybe a graph error that comes from Terraform core or a dynamic expressions core, you could separate it and say TF log core. If you're debugging something because maybe you wrote a provider and you want some more information, you do TF log provider. Right? So, that's debugging. Mhm, mhm. Let me pull up the link though. I'll put it in LinkedIn so you don't have to keep swapping back and forth. Here we go. All right. So, what is this key takeaway? All right, well, let me give me a second. Let me post this in. Okay. Um so, the key takeaway to this is that if you are trying to go do some work with deep in Terraform, you've got an error message, you're not really sure what is going on, um you can always enable TF_LOG and dig through it. Um some other things that we're doing right now, we're just sort of meandering along. We started with a back end uh and we started playing around with back end and digging through some of the code. We're not actually going to edit the code uh because again, compiling Terraform on a local machine can get a little challenging. Uh but for at least for now, as you're resolving as you're using a back end, we'll probably be using a remote back end going forward. Um we also played around sort of with lock file. We played around with the remote back end, how the lock file works. That will be differ it will differ depending on the back end type. So, S3 creates a lock file and then uses S3 to lock it to um do uh some to basically update the lock and update the state. I'm using the commands. Mhm. How did this work because I didn't uh specify the like AWS source version in my Terraform? Yeah, so it automatically initializes it. If you don't initialize the provider, um Terraform automatically go download it for you even if you don't define a provider. I use still use I think AWS like your default AWS region? Uh your Wi-Fi's cutting out again. Ah, not again. Uh all right. — Try it again. So, you need to in a provider. Provider automatically gets pulled down by Terraform. So technically, the only reason why you

define provider really is to find the region. But it's good to define the provider binary just so that you can pin it to a version. Otherwise, if you don't version, it becomes uh you know, a little bit problematic. So actually, we could probably dig through some of the code. So we could go through the go back to the Terraform directory. We could dig through some of it. I'll have to do some searching, but uh It is internal Terraform. I think eval context. Uh Oh, there's architecture. md. We could go look at that. I don't know how we're better to read this. I think that Yeah, this uh is I tried to open my web browser to like a local host. Okay. Well, this is nice because it does actually explain the architecture of Terraform. Um which is kind of you know, we could actually dig through it. Rather than that, do the code cuz it's a little bit dense. Uh let me pull it up. I don't know if you can Mhm. You know what? Let me just send you the the the the browser. browser. There you go. There's the browser version. Have we put any Easter eggs in Terraform? I don't believe so. But, I'm not sure. Where did you send Oh, sorry. You sent on Slack? Yeah, let me delete that one cuz you know what, let me put it on a comment. That might be easier. Well done. There you go. Now you can retrieve it. Okay, yeah. I'm not sure where my Slack will open up and I'm afraid it will do it while sharing my whole screen. Yeah, it's okay. Oh no, what happened? Cuz uh I think it'll maybe easier than showing this. Yeah. Uh so for the person who asked about Easter eggs, I don't believe so. I can't remember if we did. I just don't remember. I don't think so, so. Okay, this is actually pretty useful. Cuz if you do want to know what's going on with Terraform internally, there's a handy-dandy architecture diagram, which is great. Okay. All right, so we can kind of scroll up to the graph. There's like a graph. Uh there's a graph of how the request flow works. Not the graph itself, but the request flow. Scroll up up. It's a picture. Yeah, that. Okay. All right, so effectively it's kind of like what happens here. Oh. What I'm going to send uh I want to send a comment because that email context Huh? — It like explicitly links to that you can't see it, but like local host. Oh. Oh, yes, yes, yes. That's because you can pop up the docs. Uh yeah. Oh, okay. It's for folks who are developing on it. Uh we had I'm going to pop my camera back on cuz I think it will may stabilize. Uh because we had a number Fun fact, of Terraform engineers who worked remote and would like nomadically roam about. Uh, and you know, would like to do this stuff locally. So, then you'd have them a lot of them like would develop offline. Uh, you know, and then commit make commits and stuff. So, they had to have the docs offline and local. Got it. Yeah. That makes sense. Um, okay. So, uh, a couple of things you have the CLI command package, very, you know, it help it helps to know. Uh, and then there's a configuration loader. So, config is for any kind of Terraform configs you might have. These are more important if let's say you define like a separate provider endpoint or something like that, a module endpoint. And then you have a back end. So, we talked about back end. Um, which is the implementation for us to initialize a remote back end, and then there's the state implementation. Um, and that's the schema for state. So, all of these things go into context, Terraform context. Um, and the context captures it, processes it, process it through the graph, and the graph is where it sort of walks through all these references. So, there's an address for every resource, module. Uh, and it builds out a graph for it, and then it traverses the graph. That's where this context graph walker is. And then it evaluates every node. Um

I'm not going to go deeply into nodes, but for every node there's a set of resources. Uh, and so, it will apply changes to those resources based on those sets of nodes. Yeah. So, those That's the overview, request overview. Uh, request flow overview. I'm not going to go through CLI cuz there's a lot of I mean, it's CI. No. There's a lot going on. Um, but back ends, right? It talks a little bit about how each of these back ends are load are loaded in, um, how local back end works versus the remote back end. We already kind of, you know, dive dove into that. Uh, configuration loader is kind of we didn't really talk about configs, right? But configs are for HCL, right? As well as any other configurations you might have. Um so this is like kind of like the Terraform block. This is the provider block, everything else. Um again, when you notice the order of operations, it's kind of important to recognize that some things just can't get those references, right? Those addressable references. So back end can't because it's initial it's initially in part of this first part of that config uh not the config, the first part of that initialization flow that goes into context and some other configs, right? Can't necessarily be loaded in either. Uh and then there's the state manager. So this is separate. The state manager is going to create the schema. So this is where you see that JSON. You know, you can make edits to it, I guess, if you really wanted to. We're not going to. Uh and then the graph builder. We did briefly show the graph before um because we have multiple resources. Um and we showed like the digraph uh of what resources are where. More importantly though with the graph, I don't know if there's an easy way to show the graph uh like a detailed graph. Maybe there's a way we could do it. It might have some interesting information here on how to do it, but I only know how to print out using Terraform graph and then it will show sort of the dependencies. But there's a lot of uh transformation required in order to make those changes. Um and I think the more important — why are those like why are those transformations needed like along the way? Like why not leave it in just HCL the whole time? So it could be in HCL, but it needs to be transformed into a vertex, right? Into basically the graph basically needs to be transformed into a point in the graph, right? It needs a little bit more information. Um it as it stands, right? HCL is just the interface. It's just the language by which you express it, but those dependencies ultimately need to be transformed into a graph point for lack of a better word. And so, in order to transform that into a graph point, there's additional metadata. I think if you right-click on like config transformer, you could see you could This is like just the resource. So, every resource becomes a vertex. So, if you open that up Oh, github. com there. Oh. Internal share internal Terraform. Yeah, you can ignore the config. Maybe they changed the path. Mhm, hold on. I have to look for it. Mhm, config transformer. Oh, there we go. Here. Internal Terraform transform config. go. So, if you go to internal Terraform transform_config. go. Oh. This one? Transform config? Yep. All right, so this is the config transformer. And this is what basically taking the resources and then adding them to a config as a sort of config vertex. Um no, variables add outputs and providers must be added by other transforms, and there's a couple reasons for that, but basically that's uh of like the second step after initializing back end and after that you do providers, variables, and outputs. So. Uh Mhm. I like the naming pattern of can you fake transformer old? It's like can you trick transformer new new new final final. Yeah. Which I mean, I guess we could go deeply into this, but I don't know if there's a point to going deeply into this to be totally honest. But hey, now everybody knows. Um there's also provider transformer. Like I said, there are some other points there are

some other points things that you do have to uh in terms of the order of operations you do have to know. So, back end gets initialized first, providers, variables, and outputs get initialized second. Uh and then the config transformer happens third. So, that's why it's — This is one of those things that I just saw that I really liked about Terraform the first time I tried using it is that like So, you were you showed me where config transformer was, right? It was under like internal transform underbar config and then you'd mentioned provider transform and like with my simple like lizard brain I was like, oh, I want I bet it's transform underbar provider. — Yeah. Mhm. And that's like exactly where it is. And it's like that kind of like consistency just makes it so easy. Like even if I only like 30% know what I'm looking for, it's very easy to find. Yeah. Exactly. It's well you know, it's well structured. It helps a lot. You also have transform provisioner if you decide to use provisioners. There's modules, there's all sorts of things. You can even look at there I think there might be a variable one in here, too. Let me see. Yeah, variable transform variable. So, you've got like basically anything that's HCL gets transformed into part of this graph. So, this is the set of functions that would transform anything that comes from, you know, any sort of raw value or input value or anything Uh into part of the graph. So, it's important to consider that to maybe like if you're not at your day-to-day, but if you're curious, you can look into it a little bit more. All right, back to architecture. Anything else that we want to talk Oh, yes, graph walking. Uh again, we talked about graph last time. I don't know if we want to create a couple resources that depend on each other just to show the graph. Um but it does use DAG, so directed acyclic graph. — We do have two Oh, this is a data that's on a resource. Yeah. Uh Yeah. I think we have to copy it probably from the last one. Or we could make it simple, I guess. Yeah, someone's mentioning the graph feature. I was interested to see if there's a way to build out a graph of all of our Terraform modules that we have to understand and our Terraform relationships. Yeah, it's a little bit trickier. It is probably possible to do it by Terraform the Terraform graph function. I just don't know it has probably more information than you want. It is quite verbose. Um Okay, so now I have two We have two resources that one depends on the other one. Okay. Give it a better name. Yeah. And it's just Yeah, the idea of the first one is injected. Okay. Let's see if this even applies. I don't know if it applies. Might not apply. Why wouldn't it Oh. Yeah, do six auto. Just Uh no, you can change You can update it. Just do six auto. Yeah. Six auto. Mhm. Yeah. Oh, we do still have to upgrade it. Uh it should be fine. Oh, yeah, it's fine. Okay. All right. So, you could do terraform after that applies, you could do a terraform graph, and then we could see. Um I think that's the funny thing about the graph. The graph is giving you quite a bit of information, but it's up to you to parse out what kind of relationships are available. The other option that you can do that it do is I think through the state file. Might have something. So, we can investigate the state file. I think the state file has some schema built out of what relationships exist, but not to the same degree as this the graph one. Yeah, if you want to download the state file, we could do that. We can look at that, too. Wait. Oh, there it is. Okay. Yeah, I don't know if you could collapse the resources, cuz the resources would just have them instance by instance, but I don't think it would have dependencies listed, right? No, wait. Um ah, yeah. Open expand it. I don't think they have dependencies

listed on each other. No, I don't think so. Scroll into the instances. Let's see. Yeah, if you scroll around in there, I don't know if they have dependencies listed in there. So, I don't think it quite work. I think you might be able to get away with it with Oh, no. Yeah, there's dependencies. There you go. So, you might be able to reconstruct it. I think on the module schema as well if you have modules um for the person who asked uh or was thinking about it. You might actually be able to do it through the state file by getting the dependencies, right? There's like that dependency section uh under the um schema. So, name is part two. If you keep scrolling down under part two, there's some dependencies in there. There's in the schema. Where is it? Yeah. So, it's telling you like that's data. aws_ami some. That might be cleaner than trying to do the graph function. You might just like parse out dependencies and then find where the dependencies are and map them that way. Yeah. So, that's another way of doing it, but we can also do the Terraform graph. Oh. Now, you have to go o4 internal. Oh, I forgot I keep looking. Oh, I got to use that uh the digraph tool you sent me. Mhm. Yeah. I don't know. You have to drag the browser around probably. It's interesting that it um So, this part two like, depends on this which depends on this, but this also depends on Yeah. this. It's interesting that it doesn't like show that second. Mhm. That is interesting. AWS instance part two. Yeah. Uh Well, I guess it's like sort of what resolves first, right? If it can. Uh Let me see. Cuz it's directed acyclic, cuz it has to be directed. Yeah, it can't have cycles. So, it probably doesn't I think it like in terms of how Terraform is graphing it, it's not going to have an edge between AWS instance part two to data AWS AMI some Ubuntu image, just because it has to be directed. You don't have a loop back. So, implicitly, if as long as AWS instance web different name is able to resolve data AWS AMI some Ubuntu image, and as long as that's capable that edge is resolved, then the AWS instance part two only needs to depend on AWS instance web different name. So, you're not going to show every single dependency, I guess, as part of the DAG. It's a direct relationship. If you remove the dependency for AWS instance web different name, though, then it would point AWS instance part two to data AWS AMI some Ubuntu image. Yeah, I was trying to picture a I guess if there was some other resource here Mhm. that depended on that like introduced a cycle. Mhm. And it was because this depended on Ubuntu Yeah, so if this Yeah, I'll do I'll do it as homework. I'll see if I can do it. — now. I mean, it's like, what else are we going to do? We're Are we You promised we would do it in the most inefficient way possible, so. Yeah, I know. I'm trying to think if there's a way to like could I introduce another resource that Ubuntu image. I was trying to think of like what would make this very hard for me to debug if I had a circular dependency of like of some other resource here that like this depends on and also this one depends on? I think — like when you have a circular dependency, it tells you like the cycle, right? And it prints it. Yes. But the cycle like would say like uh sub Ubuntu image and then instance part two and then like let's say the this mythical fourth resource and looking at this it's like oh well like I'd have to trace that like understand that like this arrow implicit this arrow implicitly includes like

all of this one. Or a different example is like let's say there was some other resource here that only this EC2 depended on. Mhm. Right? Then like it's not clear that part two depends on this and also this but not on this other one. Well, you could try it. So if you do another AWS instance Yeah, and then uh you know, you could hardcode the AMI or something. I don't know. Yeah, so So you could do AWS instance and you would do like zero instance or something, part zero instance. I don't know, part one. Yeah. Okay, but I want — Oh, you already have one. Okay. So yeah, so you could hardcode the AMI. Well, I want to keep it depending on that I I got one second. Uh Okay, so I'll create a second Ubuntu image. You can do the same. It's okay. It's just a different ID. So, you just reference second Ubuntu image. It's okay if it's the same value. Yeah. Okay. Huh? Okay. I'm Because I want it to depend on both those, but you can't give it two AMIs. Okay. All right. Yeah, I guess. Uh okay, yeah. Um So, this will just point this instance to both AMI It's to both AMIs for now, right? And then you can like I would apply this and look at the graph and then before you make more changes. Then you can understand what you're looking to do. Yeah. Cuz which I think what you're trying to do is introduce an implicit circular dependency, and I'm not sure this is the way it will get to it, but we can see if we can try to get there. I didn't say it was about implicit. It was just that like I would expect using this to be able to like uh if I had a circle like try to visualize where is this coming from, and if I had another resource here that like also depended on uh So, if this different name depended on this image and some other thing, it's not clear to me that part two like it depends on this and also on this, but not on this other one. Like this other one can be deleted and it'll have no effect. — Well, the graph The thing about graph though is it's not showing all the dependencies. It's just showing how Terraform is going to walk the changes, right? The It's basically I think that's a misnomer about graph and probably why dependencies from the schema is a better way to parse the actual dependencies. Um cuz I wonder if you look at the state file that you pulled down if does actually say like the AMI for part two or whatever. Right? But a graph isn't the diagram this graphing isn't going to show all the dependencies necessarily as much as just show how Terraform intends on walking uh the changes across the graph. Got it. And I think that makes a lot more it sense. This is less of like an explicit like listing of all of the dependencies and more of a this is what Terraform is like discretize those into like basically all of the leaves Mhm. of this can be done uh Well, it's not tree cuz there's multiple roots. But like all the things that are like the very end of this can be done first. Right. And then you can go to like whatever is next and then whatever is next. That's it. Yeah, okay. And like the reason this one here isn't like down here and next to this is cuz this also depends on that. Okay. Yeah. Got it. So it's less of a dependency graph and more of a — Right. deployment step graph. Yeah. Now, if you had AWS instance part two, if you change the AMI for AWS part two and you pointed it to second Ubuntu image, the graph might change. Because now it's no longer Yeah. So if you did AWS instance part two and you change the AMI to second Ubuntu image. Oh man, someone at AWS is going to hate us if we keep tearing like building and tearing down these instances. Oh, wait. They did not do anything. Oh, cuz like it resolves to — same AMI. It's okay. But you want to just print out the graph cuz effectively it could be different

right? You don't know for sure. There you go. So, that effectively would say like, "Okay, it does need that image. " Right. So, the first res the first node would be some Ubuntu image, which allows you to traverse and create changes on AWS instance part three, as well as AWS instance with different name. The second Ubuntu image is a dependency for AWS instance part three and AWS instance two, right. Got it. Cool. Mhm. Yeah. But the insert of in terms of like the way that the graph would walk, if anything is if a dependency is the same, it's not it's going to point it in a directed way, right? Uh it's not going to it just show every dependency. It's only going to show how Terraform intends to walk the relationship. Yeah. Mhm. Yeah. All right. So, we're coming up on the last of internals unless there are other things you want to look at. No, that's all I had. Uh there's I mean, we could I guess we could do provider metadata. I'm trying to think if there's like one last bit of it, which I guess we could talk about. Um Uh we could talk about parallelism. I guess Um So, you can uh you can when Terraform walks the graph, right? And you could go back to the graph that you showed. Um it could walk multiple paths at once, right? So, you can tell it to walk many paths at once, as many paths as possible as once if uh if it's possible. Um and I think the So, uh based on this, the default is 10 nodes. So, up to 10 nodes in the graph can be processed at the same time. Um and so, you could do that using the parallelism. Uh it's not typically something you would want to do. I mean, unless you decide you It's not normal. Like, you would probably only want 10 at once anyway. But, if you want to like do rate you know, if you have an a provider that rate limits and you want to control how many, you know, API calls Terraform makes at a given point in time, you might change parallelism. So, we could do parallelism as one. And that might do something interesting. I mean, not that it does anything. I guess we have to make a change to some of this stuff. You could destroy it. You could do destroy in parallelism. Uh it's space-parallelism. equals one. So, this means we only do it one at a time. Uh one R. Two L's on the first set. And then one L on the second set. Yep. I was way off. What's that like a That's okay. — 50 90? It's like, is this one or two? Uh and uh I was wrong on all three. Mhm. I think I've only ever used parallelism once, and that was because of rate limiting. You could do yes. So, it will theoretically Yeah, so you'll notice it only does it one at a time, right? — Yeah, cuz it used to have like all three of the instances say like destroying destroying, and then, you know, in 10 seconds it would say just like yeah, still destroying. And I was just those one at a time. Yeah. So, parallelisms — this the cloud formation destroyers. It takes all the time. — like 100 of them, I guess, if you really wanted to do parallelisms equals 100 if you wanted to make maybe make things, you know, if you want to spice it up or something, but um I have only really used parallelism in the past to uh to like decrease the number of resources that are being created or destroyed at a given point in time uh from a rate limiting perspective, but I think if you have I think this is a good use case where it's like you send one API request and then you're just like basically waiting for it to finish for a long time. That like if I had like 1,000 instances, I wouldn't want to do them one at a time or maybe even 10 at a time. I would say I want a high degree of parallelism because the number of resources doesn't directly correlate to how much load I'm putting on AWS's servers in terms of like my API calls. Whereas like if I'm creating S3 buckets or creating instances, that's probably a different story. Yeah, exactly. Yeah. Yeah, this is I now we're just going to have to wait to be destroyed. Oh, no, it's still in the lock file. Yeah. Now everybody knows. Uh everybody knows now uh that it exists. Uh all right, and then uh do we talk about JSON output? I don't

know if we talked about JSON. — think we did. I think we only had very briefly like we output things as JSON to look at it, but I don't think we went too much more into detail than that. — Okay. Well, I mean, uh we I think in part one we probably saw it, but basically you can do a JSON representation of a plan and an apply, and that's the way you would uh you know, you could parse it, right, for various things. I don't know if it's worth do you it do you want to go back and do it? It's up to you. The JSON outputs. Yeah, we have time, right? We have Yeah, we some time. Yeah. That's Okay. All right. So, uh let's go to the Terraform 04 again. I guess it's still destroying. Oh, now it's destroying. Stay here. Um we do Terraform plan, and then I think it's like out file or something or out Terraform. Yeah, Terraform plan space {dash} out {dash} uh {dash} out equals and you could do TF plan. Something like that. Yeah. And then hit enter. So, then you'll do Terraform Oh, after this done, then you do Terraform show space JSON. JSON and then uh {dash} JSON. Sorry, {dash} JSON. JSON. Uh and then uh TF plan. Okay. Yep. Okay. Well, I guess we could jq it or we could write it to a file. And then we can dig through it that way. Even better, send it to some random website. Uh Oh, okay. Well, okay. All right. All right. Just make sure your credentials are in here before you do this. Oh, that is true. Yeah. And it's like Click the big box. Click the box toward the right side. Not the box, the little arrow thingy. Yeah. Make it big. There you go. Okay. All right. Um so, what we just did was we did a Terraform plan and then we wrote it out to a binary plan file. Uh and then we said, "Well, we want to understand what's going on in this plan, and we want to parse it. So, we'll do Terraform show JSON and then the plan file. " Why would you do this? This is so that you know what the plan is and it's in a JSON format that you can parse it from for another tool. Hint, anything that is pretty much doing testing, anything that's doing like policy as code, including but not limited to Sentinel and Opa, would be parsing this JSON file. So, it's a bit of like a cheat code for any like testing or any security testing that you want to do on changes that are policy as code parsing that you want to do on changes to Terraform. And the Wiz run task also, I don't think it directly gets the plan like this, but something very similar in a run task. Yeah. Um and really the most important thing about the plan schema is that it shows you the plan values, um things that are maybe not going to be They're not, you know, you're not going to know until they're applied, right? Are left blank. But things like, I don't know, the AMI ID has been resolved, so that's been, you know, that's been added in. Um force destroy, false, you know, some of the defaults are there. But other things that it just don't exist, you know, fair whether it be sense values or otherwise, uh are just sort of left blank in the plan. Um if you have like they These are create, I think. So, there it will tell you like these are create under create actions. So, you'll they'll create these instances. Um yeah. And so, these are things that you can also parse. There's also resource changes. Resource changes kind of tells you like what the action is. So, it will create this instance. Uh and so, this is useful if let's say you have a policy that prevents a destroy, um you can parse for anything that's like resource changes change actions destroy, and then prevent that from destroying. So, you know, you can do a lot of You could parsing with this crap. Uh you know, this there's there are a lot of options here. You can do a lot of fun things. Cool. Yeah. Um I'm trying to think if there's anything else that's like uh state is the same, very similar. Um I'm trying to think if there's any other JSON output. There's also like provider config. There's all sorts of JSON format representations that come out. Um you can also — of those tools would want they'd want both plan and state, right? Cuz like in order to do like policy as code Mhm. kind of thoroughly, you need to know like not just okay, you're adding this thing, but like I need information about the thing that you're adding it to. Like it's might be fine to have a policy so long as it's only applied to buckets that block public access or things like

that. Yeah, exactly. So you could use any number you use you would have to use a lot of logic to basically parse through this file. Um but yeah, it's possible. And uh if you are familiar with Sentinel, Sentinel does the same thing. It's just retrieving it out of this out of a JSON representation. Uh All right. So the next thing that I will probably say in terms of JSON is that you in terms of advanced or internal fun internals, um Terraform is based on HashiCorp configuration language, which is its own sort of domain-specific language. But you can also use JSON to define Terraform. Um so it's useful if like let's say you wanted to wrap around some kind of like I don't know, programming language or something, you can use the Terraform JSON representation. You're not limited to writing it in HCL, right? Um So I could have like my PHP and then JSON that I use to do a Terraform apply to? For better or for worse, yes. Uh it I don't recommend it. I really don't, personally. Um but you know it does happen. Uh I mean, do you want to do that? I don't know if you feel like you want to do it. I have zero interest in doing that. I'm just saying someone could. Oh, yeah, yeah. I wouldn't recommend it. Personally, Um you know, it does happen though. Uh okay. So, I think the last bit of this and we'll end early today because you know, sometimes we we see what we feel. Um Oh, I had a question. So, um I don't remember where I was or what I was doing, but I was Googling. I was trying to find something for HCL and maybe this was just like an LLM like gaslighting me, but like was it ever called like the Terraform language and then became HCL or vice versa? No, not that. Okay, so this is just like LLMs just — Maybe it is. I mean, maybe. Perhaps it was the Terraform language and then became HashiCorp configuration language. Uh but I would say like it would probably it was probably always HCL to begin with cuz uh Oh, here it is. Uh this is probably what I found. Oh, wait. Yeah, yeah. You got hallucinated this cuz the Terraform language is indeed a Terraform language, but it is built on HCL. Right? So. Got it. Okay, okay. Wait, so does that mean Sentinel like Sentinel's also built on HCL, but it's not the Terraform language, right? That's correct. Okay. Uh I have a much better mental model now. — Right. Vault, Vault console, and Packer even are um are able to be expressed in HCL, but they are not Terraform, right? So, it's not the same syntax. They're all like sibling languages to It's just the format. Like the format is like you know, I think only unique to Terraform is like that resource uh instance and then, you know, the blocks. Like those things are unique to Terraform. But in terms of like the squiggle brackets everywhere and things like that, that's not unique to Terraform. So, if folks are confused, you know, it's HCL is a generic language to express all these things. — Well, yeah, I was trying to find a I think I was trying to find like an LSP or an extension for Sentinel policies. Yeah. And I was like, this looks like Terraform, but like it's clearly not and like the Terraform thing didn't pick it up and I was like, yeah, and that's what landed me on this page. I was like — Yeah. Did I hallucinate? Like what that was called? I don't — no, yes. Um it used to be that we would have an HCL plugin on Vim. When when Terraform development was done like done back before all of these language servers and like IDEs supported Terraform, uh we would do it we would install an just like an HCL generic plugin for Vim. Um and it would format HCL very generically, uh which is why Terraform format exists, to be honest, because uh it became clear that Terraform had its own sort of like required formatting um outside of just using like a generic HCL plugin. And the HashiCorp employees running Terraform format like every 10 seconds just like juiced the numbers to make it look like it was super popular. No, I mean, I don't we've never tracked even like the subcommands. There are so I'm like I don't know, but yeah, everybody loves a good Terraform format. Uh all right, I'm going to check questions in the chat. Oh, I did find like the equivalent of that in CloudFormation yesterday where um I was trying I had to like manually inject the value into an existing template. So, I used like a like the YQ, which is like the YAML version of JQ. Yeah. I just like

injected it, but it like loads it, adds the thing, puts it back. So, it like auto formats the file for me. So, it got rid of my trailing spaces and cleaned it up and I'm like, I'm going to use this more often now. I mean, not more often cuz that would imply I'm using CloudFormation more often, but like when I do use CloudFormation, I'm going to use this. Nice. All right, I'm just checking chat cuz there's some questions. — Yeah, so many asked but oh yeah, the um how does he go routines to run things parallel and still follow the dependency graphs that we had? Yeah, so uh in short, let me pull it up though and just so folks have it. Um I'll pull up the here, I'll put the link here. So, in short parallelism will only follow the it this is the resource. This is um a description of graph as well as like the plans. Uh there's like a there's a nice presentation uh on how graph theory applies. But uh the important thing is that as it traverses the nodes um it will basically um only validate the direction the direction, right, of the node being traversed. Uh and so when it's walking the graph, um it will look for any of the independent nodes. And so when you use uh parallelism, it will set a go routine depending on how you're setting how many what is it operations you want to have done at one point. Um and so it will only look for it will only walk the node path uh that are available to it, right? So, if you have only three and you know, three node paths, I'm are they're not called paths, right? Graph theory. Um if they're only like three paths that the go that Terraform can comfortably walk to make those changes uh in parallel, it will only do those three, even if you set parallel to 100, right? Um and so that basically means the go routines are tied directly to how many paths that it could walk at any given point in time. I think it's an upper bound. Yeah, It's an upper bound, which means like if you created 100 resources and none of them were depending on anything. Like if you said AWS instance for each 100 of them, you know, or count 100 of them, then and you set parallelism to 100, then it would create 100 in parallel and that's 100 go routines, right? But um it's not going to uh it won't if it if there if you have a many like a if you have an instance that depends on something else, it will only create uh the resources in that go routine in that path, right? So, it won't do them all at once. But uh with parallelism, don't try to try to don't try to push it, you know? 10 is probably the upper limit. Oh, you're playing with the yeah. Oh, okay. I'm playing I didn't know what the engines were, so I figured it out. And then I remembered I'm sharing my screen. Oh, okay. It's okay. You can play with it. I use dot usually, but circo is kind of hard to see personally. Kind of anyway. Yeah. Yeah. Um I think the last bit which was on my list was remote service discovery, but it's not something that I would use day-to-day. Um and I you don't have to show the screen. I'm going to actually pull the screen off and then just um pop the link. Uh remote service discovery is we'll probably talk about it much later in the series when we talk about provider development. Um it's for like if you have a provider or a module that you want to reference and you want to discover it at a specific uh endpoint. Um there is a way for you to specify that in Terraform. So, Terraform by default goes to the public registry. Um you could also add uh like modules or providers um you know, based on like an external endpoint. So, you know how Richard you downloaded the AWS provider automatically without ever actually declaring it, it's using remote service discovery under the hood. So, Terraform is basically pointing to the public registry endpoint and discovering any valid list of providers that might relate to what you're looking to do. Um, with that resource AWS instance resource. If you are in an enterprise and you have your own registry and your own endpoint providers and modules that people should be using. And so, if someone is working in Terraform and they wanted to do remote discovery, you could point the remote discovery service discovery endpoint to your internal endpoint instead. So, if someone doesn't declare a provider specifically a provider source specifically um, you can basically force them to default to a service discovery endpoint. We're not going to do this

but Okay, I assume there's some kind of security mechanism on there so that like let's say I'm like the security team at Big Co X that I can't just like fake that like the whatever URL was or endpoint that was used to do that resolution, I couldn't like fake that to redirect to like the internal one so it forces everyone to use the internal one like — Yeah. I assume there's at the very least some kind of like SSL that's like these certificates don't match. This seems very Right. Yeah, so it has a discovery URL so you have to make sure that as you do it your discovery document has to be implemented on your internal endpoint. If there's no discovery a valid service discovery document implemented, then Terraform will fail, right? It basically will say like you don't support any resources and there's also credentials. So, you can valid you can you know, as part of discovery can force it through a CLI config through some credentials as well. Um in terms of the there have been like questions about like how do I validate like the sign the binary, you know, if it's a signed binary things like that. Uh Terraform does do it but uh only through the public only those that are gone through the public registry or the Terraform registry, right? Uh Terraform private registry. Uh if you're implementing your own registry, you have to validate that those are your signed binaries and things like that. Yeah, cuz the idea was like it's your registry, you should be validating that. Right, yeah, it anyway. But effectively um this is really like I said, I I'm not going to go through this today. Partly because it's a very advanced usage. If you are someone who wants to uh basically force Terraform to default to a different service discovery endpoint for modules or providers, then this is you know, this is the option for you. Yeah. All right. So, I guess that we're going to call this the end of it. I mean we ended early today. We did a lot. We should recap. Would you like to recap? Uh yeah, I think that my biggest takeaway is the uh like the back end is instantiated before the variables like because the you know, we always say that like all of the stuff in the directory is collapsed into like ostensibly just a big file, right? And then in run it's true and not true. It's like true true-ish uh where like some of those are like more flat than others, more first than others of like the back end as an example which I thought was uh very interesting. Yeah. Yeah, so it's like back end, providers, uh providers can resolve variables but like it's back end, variables, providers, outputs, and then the resources, data sources, and things like that. Yeah. Yep. Uh yeah, and uh we spent some time on the JSON output for like uh pulling up the plan. Like yeah, you know, after a plan or the We didn't do state this time. I think we did state the last time. We looked at like the JSON representation of that for sending it to either an observability like a cost management tool or a security tool. Uh, we went over the very briefly like roughly how the graph is constructed and how it walks it and how it treats the thing that it calls a graph. What is it actually graphing? And how to use that for either debugging or for understanding if you have room for, you know, will adding the parallelization flag make your deployment faster? If you look at that graph and that it's one long line, then it doesn't matter what your parallelization Right. Exactly. And — We have like one big edge and that's like that's all, you know, everything depends on each other then. No, it doesn't do anything. Yes, but if your graph looks like one of the armies from Lord of the Rings where it's just like a thousand ends, then yes, like that a parallelization would help quite a bit there. — Yeah, there you go. And uh, let's see what else was there. Oh yeah, we as we just went over like the remote service discovery. Like it pulls in the like the effect of the required providers. Although it's not recommended because like you could surprise get an update at some point and that's generally being surprised with updates is not a good idea. Yeah. And we spent a lot of time debugging the AWS S3 make bucket command. An inordinate amount of time playing around. — Big hugs. Like I said, we are not very efficient. We're just, you know, we're very detailed, just not very efficient. All right. I think that's it. Um, I know these are internals and they didn't really dive deep like deeply into like editing Terraform for many reasons. I think it may or may not be worth it. But if you are interested in doing it, certainly give it a try, fork Terraform and mess around with it a little bit. Um you can check out the link from Daniel on uh Terraform internals. Um highly recommend it as a read just to understand how exactly Terraform works. Uh if you feel if you plan on contributing or you're just curious. Uh and then in terms of the next one, we

were slated to do the AWS provider and sensitive data. But I think maybe we should just put modules before that cuz I think we talked about sensitive data already. We did. So I think next one we're going to talk about is modules. Um yeah, that's the next topic. Okay. Yeah, I don't know. Do you unless you have different Well no I so part of I'm very selfish and I had a question about the AWS provider. — Ah. Uh that I was hoping you could answer on the stream. Okay, I mean we have time now. We could talk about it now. Well all right, so here's the so the question so there's the data AWS identity and then like current, right? And then you can use that to like it's the ADM that we use for like getting the account ID or the ARN for the role. I assume that like that data thing it's like making an API call of so like it will it's Um I could it's running like a binary or I guess it's using the Go SDK I would assume to like get that. Um and I was curious like if somebody wanted to like figure out how that was done, like how would they do how would they see like how is AWS actually getting this? Like it says AWS identity and then they I've just taken it for granted that it just returns a thing, but like Yeah. How we can do that. How does it how Yeah, we don't do that today or if you want to do that today. — can do No, it's actually not ter- it's like you know, it's a good question. It you know, we're tangent we're making a tangent, but uh it's fine. It's not actually a big deal. Um okay, so I'll share my screen. Okay. So the I brought that up is cuz like so I'll be here week, right? And we were also planning on having a special guest for modules, but I'll be gone the 2 weeks after that. One for uh uh the AWS Game Day at SRE Day in Seattle. Uh everybody who's watching, please attend. Uh and then everybody who's watching in the future, but before the 21st, please attend. Uh and then after that, I'll be on vacation, and I have no offense, zero interest in dialing into the stream from Aruba. That's okay. Don't bother. — Okay. All right. Uh yeah. But I figured it'd be easier to like have the guest on one of the weeks when I'm out. We haven't decided yet. But I you know, like it's great to have a guest, but you know, it's just more fun, you know, more fun to keep it all the same. I don't know. It's fine. Uh we'll figure it out. Um I just think there's not much to AWS provider specifically. I mean, there is, but there's also not at the same time. Um yeah. Uh let me see. Okay, so in order to find it out, I would just recommend anybody who's like curious about the internals of how this stuff works, go to the repository for the provider. So in this case, I'm going to the AWS provider. And what I do is I look for the um uh data source. So everything is kind of like labeled the way that you would want to. So I look at the instance Everything in a provider is labeled as the provider um identify the Sorry, not the the type, and then you know, what it is. So in this case, the type is caller identity, and then it's a data source. So I just look for the file, and it's pretty well organized. Uh you know, not anything I would say like, to worry about there. Um most of the files are well organized, so you can kind of just dig through it. Uh and then let's see what it's doing. So this is where anything We'll get into provider development later, but anything you do with provider development tends to have some kind of call. And in this case, you'll notice it's an STS call. So, it is making an STS call to AWS SDK Go service STS, and that is how it is getting the information. Got it. Yeah. So, anything that you have about account ID, etc., it is all being done when it by making just like the STS client call. So, that's how it's getting the identity. Got it. Okay, yep. Got it. All right, that's what I was looking for. I was trying to figure out is it using the CLI? I was like, this got to package all that. Like, that's got to be weird. And I was like, oh, it's probably just using the Go SDK because it's already got it. It is. Uh so, that's the that's a quick answer. Maybe we'll have modules next week. I don't know. We'll figure out something. But, we will have a stream next week, and maybe not the next couple weeks. We'll fig- I'll announce. And then we have one computer to do that. Will other products be covered in the internal series? Nomad is in planning. I don't know when we will do it, but Nomad is in planning. Um and I will not be in the driver's seat for that one. I not driver seat. I will not be in the passenger seat for that one. I will be the one driving while the passenger teaches me. So, to be announced. But, I

will be taught Nomad because I am not as familiar with Nomad. So. There we go. Okay, with that we are going to close today's episode. Hopefully, you'll tune in for next week's episode. To be announced, I think will be modules because it'll be more interesting. And then we'll figure out what happens after that. All right. Thank you, everyone. Okay.