Getting into Terraform, Part 6: Refactoring API Gateways & Lambdas

Hello everyone and welcome to Getting into Terraform part six episode six. We've It's been a while because we've been busy. Yeah, — I don't have been a long time. — I know, right? — No, it's just Yeah, just very busy. Uh vacation conference season starting. — Yeah, exactly. Uh so for folks who are join who have never joined us before, welcome to getting into Terraform. This is where we learn Terraform in the most inefficient detailed way possible. So if you're looking for a 101 course, you're looking for an introduction to Terraform. I'm not sure. Uh you know, we might have some tangents here and there, so it's hard to follow along on occasion. Um and if you've not joined us, uh drop a note in chat, say hello. Uh but as we do, be welcoming, respectful, and professional. We um are trying to keep our chat as friendly as possible. But we want to hear your suggestions, your advice, your uh tips and tricks because we're not necessarily experts and we want to hear what direction you want to take this stream. There have been times where we've uh taken the suggestions from the chat including deleting lock files um you know targeting something that we shouldn't target on model you know various things. So we've had a lot going on. Uh anyway, if you are trying to follow along today and you're like, "Hey, I don't know what happened the previous streams. " We do have previous episodes. Um check out our playlist on YouTube. I will post it again at the end because everybody asks in the chat. Where are your previous episodes? Are these recorded? They are recorded. They are usually in a playlist. Uh so check that out. And if you do want to follow along in a repository somewhere, we do have a repository available to you. Um check it out. We sort them by uh episode uh and what we do. Except maybe this one. This might be the bit of exception. If you haven't tuned in to the other episodes, we've gone through a lot, Richard, right? I think we've been through a lot. Yeah. I don't even — so many episodes. We've been five. Um and so we've done uh modules. We've sort of done the basics semant like the basic semantics some of the more advanced like locking state um and sort of like what you can do with terraform and we dove into providers the AWS provider for example so we've gone through a lot but today if you're going to listen to an episode I argue this is the most important episode and most relevant episode you will listen to and that's refactoring Nice. And also the most fun episode. — Is it? I don't know. — I mean, if we tell people that, they'll believe it. And by the time they realize it's not fun, we're already threequarters way to the episode. — Hello. Oh, we have some friendly uh faces in chat. So, hello friendly faces in chat. I won't name you on Hello everyone. — And I see the chat is like it's pulling in the LinkedIn comments this time. — I know it's actually pulling it in. We don't know if you're on LinkedIn. We're sorry. Sometimes we see them, sometimes we don't. It depends on if the platform pulls them in. But if we don't see them, we'll get to the LinkedIn later. There's a bit of a delay. — Okay, so refactoring. Uh, let's talk about that. Richard, do you like refactoring? — Oh, it's great. It's like uh gives me an opportunity to curse like past Richard who's like, "Oh, this is a future Richard problem. " And just like leaves a huge as things evolve and you like, "Oh, let me just see if this works. And then if it works, I'll come back and clean it up. " And then it works, say, "Yay, it worked. Let me move on to the next thing. " And eventually you have to pay for that and usually refactoring is the way you pay for that. — Yes. So, — right. Well, I guess there's different approaches for refactoring, right? Because someone did mention in the chat like refactoring does include changing to use newer code styles. So, there's a different there's a couple different reasons why you would refactor, right? The first is um you know, of course, changing to use the newer code style, right? If that means that you were on I'm sorry, anybody who was on Terraform 0. 10 10. Bless you. You were not using the rich syntax richer syntax system and things like that. Uh you know, so yeah, you didn't get the benefits of it. Maybe now we're on 15. Yeah, 1. 15. Um we had a lot of changes. Maybe you want factor to new style and take of new features. That's one that you would refer. The second that factor maintainability right if you just organically grew terraform configurces to adding resources to it and you realize a lot of duplication getting really challenging to manage and to review like you're just sitting there scrolling you're finding thousands of lines of terraform um any change that you apply the common symptom of this of the most monolithic terraform form um

Terraform pattern that you you've approached that is when you run Terraform plan and it takes 40 minutes. There have been people who have told me and my plan takes 40 minutes or my appliance longer and that's literally because we put way too many resources in one configuration. So, uh yeah, there we are. So, I those are the two main reasons to refactor. If anybody can think of any other reasons, I don't know, Roert, do you have any other reasons to refactor? Uh yeah, usually like yeah, if you're using like a coding agent and it just decides to like wipe away all your stuff and completely rewrite it in a new coding style, it's oh, it's all going to be functional now. object-oriented now. That's a refactor of sorts. — I mean, I guess this is the crux of why we are I would say I would argue this is the most important stream and most important episode that we will probably ever do. And that's because I myself use a coding agent right here. Um yeah. So let's acknowledge it for what it is. This might actually be our first official AI episode. Um because literally the point of re basically whatever you do with your coding a coding agent now is probably just going to be exclusively refactoring. Like let's be honest, you're just going to refactor a lot unless you do a really good you build a really good skill in the first place where hopefully you are using a skill um to modularize your Terraform from the very beginning, but in all likelihood you probably aren't, right? Uh you don't know any better or maybe you're just going to generate a giant mass — one big main. tf file. — Oh, at least I didn't do that for our example. — You can thank you for not doing that our example. Um but yeah that's part of the thing that you have to recognize is that now you know a lot of our coding practice uh you know is going to involve refactor especially in the case of Terraform. So what you're going to find today is while we're not going to go deeply into writing the AI skills or you know that's not really the point. The idea is that we're just going to show you some refactoring patterns. So as you go through reviewing maybe some mass AI generated Terraform configuration, you're able to take a look at some patterns. Anything to add, Richard? I feel like I forgot. — No, I'm excited to just jump right into it and start uh ripping pieces out of the Terraform configuration. — Yeah. Okay. Uh I'm just checking chat. Yes. Plugins. Very hilarious. The Terraform is a single separate plugins again. — Oh, great. Yeah. Yeah, that did exist. Um, okay, here we go. All right, Richard, show the uh giant Terraform monolith. And my audio is cutting out. Okay. Well, who knows if my internet — It's better now. I I sent that when it was actually cutting out and it got better. — Okay. All right. Let me just actually push playback down into uh my usual process, which is basically turning everything off. Quality uh auto. I can't turn that. Okay, that's fine. Uh will there be replacements of data blocks with query lists? Oh, queer lists. Um, maybe if you could clarify a little bit more on the question because I think I know what you're getting at, but I want to make sure that uh I understand. Oh, query lists. Oh, okay. Yes. Um, I don't think we're going to go through that yet. Although the query files, I mean, it's the new search import feature. So, yeah. I mean, we'll probably talk about it um with the search and import like it's for batch um batch importing. So, it's um I think that's what it is, right? The search and import. So, querying I think it's Terraform query, but let me know if that was what you're getting at. — That's what I thought it was. — Yeah, queries. Okay. All right. Um let me pop up the giant Terraform configuration. Uh I was like, how do we do this as naturally as possible, right? So I asked a helpful coding agent to tell me um — Richard has opened the read me which I'm not sure you know anybody really cares that much about to go through it but uh Richard you've not seen this code before correct — I have not and that's why I open the read me because like I assume that like the part one part two is the refactoring we're going to attempt today — of like those will be the two

refactored component components that we're going to pull out. — Yes. So, uh the the way this was generated, um I took two AWS reference diagrams courtesy of the well architected library, a well architected framework. There are reference diagrams there. Um one is EKS, one is a serverless reservation system. Conveniently, those diagrams have all the AWS components outlined there. You know, all of the ports and the like networking that needs to be open. Great. We just don't have Terraform for them. And so I told my helpful coding agent aka Bob um which you know if you hear me say Bob is not a person it is this IDE. Uh I told I asked Bob okay Bob I need you to play an architecture for me for the like we're using these two diagrams. Uh and if you notice there's a scratch folder. So I use scratch memory. I don't scroll up. There's I don't know did I commit the scratch? Yeah, scratch u that's my scratch memory uh scratch folder. So uh for those who are at least the way I like to work is that I write everything to a scratch uh folder for memory purposes. Um so the scratch actually describes the set of architectural requirements and things like that. And I said generate me a bunch for it. I don't recommend it. It takes your tokens your token mileage may vary. But let's say my tokens are very sad right now. uh it blew quite a bit of my token budget uh anyways but needless to say best readme does reflect quite a bit of what the two parts are the first is an EKS uh it's an EKS cluster the second is a set of microservices using Dynamob Aurora elasticatch uh if you go elastic if you go down um the thing that's not in here is the quantum uh ledger I took out the quantum ledger because it took way too long to create and also I was sort of like I don't know if we need to get the quantum ledger available anyway we don't need the quantum ledger so for now that's fine um and so there are a couple variables but most of them are already set to defaults so you can apply this you can terraform init and terraform apply right off of that I don't know if you did but your account — uh I did the init um just so I wouldn't have to wait to pull all the stuff in I didn't do an apply because I wasn't sure in — Yeah, with the order we apply. We'll let it run. We'll see what happens. — Yeah. Yeah, it might takes so take some time there. Um, all right. — That's a lot of stuff. All right, we'll let that run. I'm going to answer a few things in chat. — Okay. — While it creates um I don't know. Is there anything you want to look into while we're here? — Uh, no. You just say you took out the QLB. Um, did you replace it with like a Dynamo DB table or — No, no, no. It just, uh, it just has a placeholder basically. None of this actually has working code uh, directly. I'm just like creating the resources at this point. Um, — Got it. — Yeah. So, if anybody's wondering, it's about 129 resources or so, and it's about 3,000 lines of Terraform. So while I mean I can't say it's represent fully representative of a terraform monolith necessarily um I would say it's pretty close is pretty close uh in many different ways. So I know that there's a couple questions in chat about uh certification vouchers. Uh there are no certification vouchers here. Um but there are um certification prep webinars. Um and those you can always check out. I don't know if they do have vouchers or not, but uh certainly check them out. I'll put the link up so you can take a look and I'll see if I can drop them into uh LinkedIn. Although sometimes LinkedIn doesn't let me post, so I don't know. Sometimes I never know what's going on there. Um, and folks are asking for more of an introduction in Terraform and Vault. Um, this is uh probably not the greatest uh episode for an introduction to Terraform and Vault. Um, but if you have any doubts, you know, feel free to um, you know, if you have any specific doubts, feel free to list them out and, you know, we can take the feedback and see if there's some way we can solve some of the problem. Um, I know some folks do have specific problems with Terraform and Ba. Like for example, there's one coming up about like why bother using Terraform? You know, I just want to be able to create uh my infrastructure in any interface that I want to. I don't really want a domain specific language. Uh some are

also like vault. Why would we put it in a secrets manager? Um I would rather use my cloud service provider secrets manager. All valid, you know, all dependent on your use case. So, if you have a specific use case um and it's causing problems with Terraform Vault, do drop it um in the chat and comments and we'll get back to it asynchronously um and so we can at least figure out if there's something that we can help with. But we always want to hear the feedback. — Vault had a feature that like you put stuff in Vault and you can have it like push it out to the cloud service providers to their secrets manager. So you can centrally manage what's that? Oh, secrets. Yeah. — Yeah. So you can centrally manage your secrets and then have it only push it to the relevant clouds and then you can like it's kind of the best of both. So — yeah. So there answering that question. — Yeah, there are a lot of options. Uh but you know again if it just doesn't suit your use case, it case. Let us know what that use case is and what problems you're running into. Um while Terraform and Vault may never solve the use case that you're looking at, um we'd love to hear the feedback and understand what problems you're running into. All right. I guess it's still creating. So, regardless, I guess we could uh go into this. So, Richard, you know, how would you like to start to refactor — process? Not putting you on the spot at all. — Yeah. I guess like my personal like I — maybe I'm wrong. So I would think that the uh the way I would refactor it is like separate out the like the unhelpful description of things that change together are like in code together. So it's like your base like networking layer is like one module or thing or like one collection of configuration stuff and then your like stateful you know your databases your like stateful instances hosted databases is like another layer and then your much more ephemeral your things like your lambda functions your stuff that's going to change as you're like pushing out code updates so your EKS tasks uh your Lambda functions stuff like that. That's how I would think about refactoring of like trying to cluster the things that tend to change together into modules or components that like completely encapsulate that part. — Yeah. So it's basically assess a very very straightforward one sentence description is assess blast radius right. So that's like the very first uh assess the blast radius of all their components. Now we're going into a codebase in which even I have not fully reviewed because it was AI generated. So, is it slop or sloppy? I don't know. Uh, but we're really going to start looking and assessing blast radius. So, logically, when you've done this a lot, you've uh worked with Terraform quite a bit, you're logically saying, I'm going to group these into something modular, right? So, that encapsulates some of this complex, some of these more complex resources. So, one includes VPC, the other might include the Kubernetes cluster. Um and then if we move kind of upward it would be like identity access management uh database cache and caches potentially serverless and then DNS typically is sort of the highest level resource and API gateway. So um you know and then observability and monitoring components are kind of a mixed bag. I think some have heavily dependencies like if you're training on BPC flow logs in which case that's more of a lower level dependency versus sort of the higher level uh element. So kind of like figuring out what has a very large blast radius which means the amount of what how many resources are impacted if you make a change to that one um grouping of resources. Uh and so assessing blast radius is going to be the tricky part. So if you could open a terminal, I don't know if this will allow us a new terminal. to do it, but if you go do terraform graph and then um output it to a graph doot. So try greater than. Yeah, greater than sign or whatever greater than. Yeah, graph dot. And I guess we could use I think we're going to need to use graph viz again or you could download an extension. I'm told there is a live graph extension but I don't know. — Yeah, I was just thinking that so I don't have to like share a new window. — Terraform. — Oh, but it's — it's all in the same marketplace theoretically. Uh, no, no. Maybe there's a diraph. — I think it's cuz it's a fork. We don't get the VS Code marketplace. We get like the Open GSIX marketplace. — Okay. I guess we're just going to have to go to the browser. — Graph is. No, — I mean, I guess there's, you know

graphis. Graph is maybe graphis. Yeah, graphiz should be fine. Yeah. Yeah, there's a preview. — Yeah. Yeah, — Graph Viscode. Look at us installing extensions that uh — Yeah. unknown untrusted extensions. It's not like this has been in the news lately. — Yeah. Uh as much as I don't want to do that, you could there's a graphiz preview. I think graph is preview. That one is the most reliable of the bunch, I suppose. question mark. — Yeah, and it was published before all this AI nonsense took off. So, — yes. Yes, — we don't have to. — We will see. Otherwise, we'll have to do this um by CLI. I don't know. — Well, I mean, I can just go to the URL for the graph is and I'll just share a new screen, — right? Uh where is — I mean otherwise I was going to suggest Rover but I mean Rover is a little bit old but Rover is a visualizer. We would have to just download it though. — Okay. Uh — okay let's see. Uh I think you have to do like comm shift command P. open preview to side. Well, that didn't help anything. Here's us trying to be smart about it. — Okay, let me see if I can find a web- based version of it. — Yeah, there is graphis online. Here we are. So for folks who are trying to follow along, we are creating the Terraform right now, but we are also trying to figure debug this codebase, right? So we're starting from um resources that have a more minimal blast radius and then looking at the other ones which have a higher blast radius. So right off the bat, we already know that there's a VPC involved. We know that there's EKS cluster involved, but what we don't know is the blast radius of those two lower level components, and those will probably mess with a lot of things. So what we're trying to do is output a graph so we can at least visualize which components have sort of the least dependencies and maybe we can look at th those first for refactor. So when we have like 129 of these uh it can be quite large. Uh so we it's very hard to just sort of like eye it and then guess where things are. Um, but I mean we could also just start with lambdas and you know be very joyfully like refactoring just that lambda part, but it's probably better we take a look just to be sure. Okay. Let me know when you're ready. — I'm ready. — Okay. All right. So, there's our ginormous graph, our giant graph there. I think you should be able to zoom in, right? Okay. So everything kind of depends on everything and you'll notice that the important thing though is everything depends on that VPC main. So that's the last thing we're touching. Um it's like at least my personal rule is that if everything depends on it, it's probably the last thing that I'm going to mess with for at least for now. Um because the minute I try to mess with it then and it impacts everything else then you know it's very hard for me to refactor everything else. So I tend to start with sort of the right side of the diagram um in which things are not necessarily dependent on it. So you'll notice there's two API three API gateways routes three API gateway routes right one two three and uh integration and then there's like EKS nodes so roughly they're roughly separate actually which is good um so they're not dependent on each other uh the EKS stuff is separate from the API gateway stuff I' say we tackle first, but I leave that choice to you, Richard. — No, I like that because that's the I am more fluent in the API gateway stuff than I am in the EKS stuff, so I'll — be less likely to make a an obvious mistake on our first try. — But it is API gateway v2. — Ah, that's fine. It's — okay. We had a discussion about API gateway v1. — Oh, yeah. It's — okay. So, uh, the reason why I like looking at the graph first is actually it tells you sort of where the dependencies are. I start on the right side and I look at the ones that kind of have, uh, less fewer clear fewer arrows going everywhere, right? Um, and so when I start to refactor, the first thing that I want to look for is like you're not trying to with Terraform, it's very easy to I guess like do everything at once. Don't do everything at once and have a specific goal in mind as well, right? So our first goal is really to uh

sort of modularize this. Um I think that's a very good first goal, right Richard? Just like modularize some of this. — Yes. — Um sort of take out some of the duplicate code and modularize some of the things if we can. And so the first thing that we could look to modularize is the Lambda portion of this. If you go down, this uh has an API gateway. It has a bunch of IM rules. It has Lambda permission. It has a bucket policy. So, some of these things we are going to figure out if they duplicate, if they don't duplicate. Um, and then some of these things, you know, we probably can extract into a module and make it a little bit easier. So, let's just start with that. So, I assume they're all in Lambda and there maybe is there an API gateway one? Oh, yeah, there's an API gateway one, too. Yeah. So we'll start with API gateway because I think that fronts the lambda it depends on the lambda but let's try to the API gateway first. Um it's also userf facing right theoretically in this contrived example the API gateway is userf facing. Uh would you want to start with it first? Maybe not but perhaps in our case we can be reasonably assured that starting with the API gateway is not going to affect directly affect anyone. If it was a more critical piece of infrastructure though, I would be more hesitant in starting in it first. And API gateway also supports like natively within the service like the ability to have like another version of it like so as we do the refactoring we can if we want to like deploy like a new version of it ensure that works before we like remove the old stuff uh so that it like naturally natively supports um something like a blue green deployment uh which helps as opposed to like — you're trying to refactor an S3 bucket where it just like oh no I'll just delete the old one and now you're in a bad spot. — Yeah. So, that's where I feel like we might be able to do the API gateway reasonably uh well, just because it does support like some kind of traffic shaping. Um but again, you know, beware, right? If you know it is userfacing, it may not be the first thing you factor. Uh okay, so let's see. We've got a couple API gateways. The question is, uh they are kind of they're mostly the same. Some are dependent on each other. So you have integration depends on config. Uh and then let's see we have a lambda permission which allows the lambda to invoke the API gateway. — Other way around it allows the API gateway to invoke lambda. — Oh sorry API gateway invoke lambda. Sorry the other direction. Um and then what else is in here? Cloudatch log group. Uh — yeah, there's a law group. Okay. And conveniently, someone has commented aka uh Bob has commented that there's a second API gateway for the inventory microser. All right. Uh okay. So that's the good news. And then let's see. It's roughly the same. I think there are some minor differences, right? I don't but some of these eyeballing it. Uh, I hate this highlighting thing. Okay, lambda permission. Yeah, just like a quick look. Most of the It looks like these three lambdas. There's three lambdas. These three lambdas have three gateways and then basically have roughly the same components, right? the landing permission, the log group, the API gateway and the adjacent, you know, requirements for the API gateway and then the cloudatch log group. Yeah. So, yeah, there a couple. So, this is just from scanning it through just eyeing it. Um, in your case, you might not have uh as much organization. Hopefully, someone has commented it and hopefully someone has communicated exactly what these are and put them in rough order. But in some situations like people will copy paste them and put them everywhere. So this may be more complicated of an exercise. But what you want to do is find some common components. Now Richard, I'll let you decide. Do you want to use AI assistance for this? — Yeah. Let's see what happens. — Okay. All right. So let's do it. Uh so uh and in my experimentation with Bob and specifically Terraform um one thing you should do is scope your quest your prompts as best you can um keep them as small as possible. Right. That optimizes your tokens but also it stops uh you know sort of injecting random stuff uh that you know you don't really want. Yeah. Right. Um, okay. So, what we're going to do is be very very specific to as a prompt. We're in code

mode, which means we will be coding. Um, and we're not adding any skills, I don't think. Uh, yeah, we don't you don't have any skills loaded, I don't think. Right. — Uh, no. — Okay. So, we don't have any skills loaded. So, we're basically doing this without any skills, any enhancement, any additional harnesses for the agent itself. Now, if you're like, why can't you use Claw? the other ones? Uh, we can't use them. However, the approximation is roughly the same, right? The result may vary just a little bit. The mileage, token mileage bit, but the result overall and the approach and the pattern we're taking is something you can do across many other coding agents, right? Um, we are not putting any special harnesses on this. Uh, I can't speak to the harnesses in Bob itself, but we are not putting anything specific around it. Okay. So the first thing we're going to do is we're going to say uh atapi-gateway. tf. — Right. — This tell it which file to use. — Yes. — Like this is context you will need. — Yes. This is the context you will need and focus only on this context. um refactor uh into terraform this terraform into a module uh configuration into a module. Um, we probably want to put some constraints around it because very important. Um, so let's see. Uh, make no mistakes. It's not uh I'm going to um you would say we will say like prompt anytime um we need to have a variable override. So this is important because the minute you start modularizing um you want to set very strong defaults, right? so that people don't have to set so many inputs. But if you do need to add a variable override for something for the module, then you want to at least make sure that this is a valid override, right? Okay. So, let's see where this gets us first. This may not get us anywhere, but we'll see. Um, we're restricting this to the API gateway Terraform. Now, if there are other dependencies across other files, it might need it. So, it might ask us about this, but it does break it down. Uh, and so it's going to create a new modules directory. That's fine. Um, we're going to create a local module. I don't think we're going to put this to into a registry anytime soon. Okay. So, it's going to generate a bunch of variables. Um, which is fair. So, we've got the API gateway and it's generating. Ah, yes. So, good question. Someone asked, what ID is this one? This is called IBM Bob. Uh, it's a coding agent. um in an IDE. So uh it's pretty much just VS code but um it has the enhancement of the uh coding agent IBM bot coding agent. You can check it out. Uh light has said before um your mileage — it's code OSS I believe is the — marketing friendly term for it's the it is a fork of the open source thing that VS Code is also built on top of. — Yes. There we go. Yeah. — Uh I need to confirm the variable overrides for the module. It says based on the current configuration. I can see standard cores. Uh — ah so this is important. There are actually four APIs that it's we're point is pointing to and some of them are slightly different right. So config, inventory and booking all have standard cores but booking has some differences, right? It has shopping has just a limited one. get post and options and booking has a jaw authorizer with Cognto. So that's good. So this is good. Okay. So it's basically asking us what do I do now? — Uh I'm going to let you have a chance to read it, Richard. — Yeah. Um I would say it should default to what it's calling the standard course and then accept a like variable that like specifies JWT. M — it's uh it's interesting that it's like shopping I think this might have been a hallucination on the first time of like shopping supports post but not put for course like it seems weird that like from a cross origin like a external origin I can like create a new thing but I can't like patch a new thing — I think — well like oh put put's generally like oh yeah his patch is also a verb uh but put is usually used to like update thing that — that's true — in place, right? Um, which uh I'm curious if that if there's like a business case for why th those two were. — Well, there's if you go to the assets, there's the original diagrams. — Oh, yes. — So, if you go to assets at the top, the

original diagrams I loaded in. Like I said, I just sort of was like, we're just going to do this uh because we can. Yeah, it was a PDF, by the way. Cool. Like I just loaded the diagram and PDF in. I didn't even have to convert the visual. — Uh okay. Shopping. Uh shopping. There you go. — Okay. Lambda. — It doesn't say much. — Uh cuz usually Yeah. when you like — right for cores it's generally with the API gateways it's because like API gateway is you've got like some domain name on the front but you don't back have to worry about like what domain name is attached account so you allow the cores — and I do wonder if that was just a random uh — hallucination in the original one because it doesn't I mean this also doesn't say much about like — how to configure your chords — right — um — I did something very interester. Hm. Well, we can just make it standard, I guess, and then ask authorizer. I mean, until we figure out everything, you know, like in theory with refactor with every refactor, you probably want to preserve the existing stuff. So, — I in my opinion, I would say we keep the limited course just because if we're deploying it, we don't want to break it. Now, if it's already broken, then you fix it, right? Um, at least that's in my opinion. That's just like I don't want to in add changes. I want to keep everything when I refactor roughly the same as before and then yeah sure if it's already broken that's like a separate fix but I would just keep yeah with limited methods uh and then keep the and then pass the jot authorizer as variable. All right. So if you were to do this manually without Bob or without a coding agent, this is something you might just find, right? You might just have to find the patterns and figure out what is available to you and then refactor that way. Um we're using this sort of night in sort of a helpful development perspective in that at least is doing some pattern matching for us. So it's able to identify these patterns and then sort of extract them. So I mean does this make it a little faster? I guess so. Uh, did it stop creating that before we start running it? Before you start running it. — Well, no, it's just like formatting and validating it. I thought that was part of its like generation process. — Yes. So, it did extract it into a module. Uh, and it is now upgrading the Terraform configuration. But I will, uh, before we get too far down into it, um, let's just open up the main and double check what it's doing like let's Okay, so it is creating one API gateway API one API gateway stage. Uh go down with access logs enabled API integration API route uh API v2 gateway v2 route. So all of this stuff is being created. Okay, this is good. It's got a count because it needs to toggle on and off the authorizer. That's good. So for the case of not shopping, the other one that has a standard course plus jot off, we have a toggle that basically allows it to be enabled or not. That's promising as a pattern. Um, and then everything else is pretty consistent. I think that the statement ID allow gave me good and then the action lambda invoke function. That's yeah, those are consist. I would that's okay to be hardcoded. I wouldn't um yeah, I wouldn't concern myself with that too much because those are things that are consistent. So, we're trying to basically put everything as much of a default. Um, the one thing that I don't like about this is there is no Terraform file. Again, we didn't put any other skills to this. So, there are no other harnesses. So, there we expect some things to be missing. But when you're developing modules, what we learned before was that it's good to put a Terraform um file. Uh, and what it will do is pin the versions. So let's put a Terraform file in just to pin the um AWS provider. Yeah. So we probably want to just copy it from here. Um we with some subtle differences, right? Uh and as you recall from our module, we don't want any of the other Yeah, you can delete that provider AWS. We don't want that uh for a module. And then uh we don't really need the other providers, I don't think. So those can go.

Okay. Um I think we've got an extra bracket somewhere. And then for a provider we want to be greater than or equal to. Yep. So what that will basically say is we've tested this against provider 6. 45. um anything that you want anytime you want to use this module as long as your root configuration is greater than 6. 45 then you know feel free to use it. So there we go. Okay. So now we've gotten this module. Um I guess we could finish formatting and validating I suppose. — Uh — let's see what — I don't know why it's validating this because we didn't actually did we refactor this to use the root module. Now I'm just confused. Hold on. Pause. Pause. Pause. Pause. Uh, okay. I don't know what it's doing. Before we go that far, um, show me what's API gateway. tf. Did it start refactoring that already? — Oh, did it pull it out of the original one? — Yeah. Did it already do that? — Yeah, it looks like — Oh, yes, it did. Okay. So, it took a couple steps. Took a couple steps forward. Uh, okay. Pause. It went too fast. This is one of those things where you just got to be like very conscientious. Uh sometimes like — it's like watching a model. — That's what Andre says, you gota just lean into the exponentials and just uh feel the vibes. — We're vibing. Okay. So, a couple of things. Um it does have the cores allowed methods. That's an override. Fine. We have module booking API. Uh yeah, so all these things, fair enough. I think they look fairly consistent. — Uh well uh no — no wait why the reservations. — I thought yeah I thought uh shopping was the one with the JWT. — Yeah. Yeah. Shopping. No it wasn't shopping. It was the other one. It was uh booking or something. — Was it? — Hold on. Scroll back up. — Yeah. Booking had the job. — Oh, shopping had limited cores. Booking. Okay. Got it. — Yeah. Okay. So uh did we finish running apply on the first terminal? It might still have been applying this way. — Oh well. Oh, it did. Wrong master password. — Oh, that's pleasant. Only printable asky characters. Uh, well, RDS cluster. It ran on my machine. So, I think it must have generate autogenerated a password somewhere. So if you go to Aurora password, if you go down, if you scroll up Oh, yeah, it did the random password. H, this drives me nuts. Okay. Um, special, you could turn uh you with special um you can override that. Hold on to force it. I always forget this port. I ran it with TFlint, too, just to make sure. Override special. — No, no. Keep special true because it's it will complain at you if you don't Aurora will complain at you. Override special and then in quotes just put hat. — Put hat. — Yeah, there you go. Okay, now run it again. — Yeah. So it will also tell us that our modules have been hold on just it refactored our modules. Scroll up. Scroll up. Hold on. It refactor our modules. Right. So now here's our problem. Right? So you'll notice here is our major problem because we refactored to module to a module. Uh now it will create all these modules which we do not want to do, right? It means it's going to delete our old resources and create the new resources. The resources already exist, right? Really what we want to do is move them in. Um, so before we do anything, let's fix the Aurora database. So if you go to API gateway, uh, and then you undo. Just undo it. Yeah, you all you have to do is undo. Yeah. Or get checkout. — That's why I was thinking about like, oh, should I just save this thing off somewhere else and then do a get reset hard? — Yeah. Yeah. Or just get checkout. It doesn't even commit it. So you could just do a get checkout for that. — All right. So I'm going tell this no to get — Yeah. Tell it no. See this is why we do this very carefully everyone. — Yeah. — Don't we are professionals who are — get check out API gateway. — Yeah. There you go. That should reset

it. Okay. And then we can always tell Bob to put it back, right? That's fine. But uh for now we're just going to uh we're going to apply it with our Aurora ch with our change to — and I can leave this module here because we're not referencing it anymore. So it's just like an ignored Yeah. Cool. — Yes. Correct. — Yeah. Okay. Uh oh, see now the outputs also changed. Go back to outputs and go fix outputs. Yeah. Oh, is it in here? Where is module booking? — Outputs. I think outputs is spelled wrong. — You get check out. — Oh. Oh, yeah. Okay. Yeah. — Yeah. There you go. Okay. So, that should apply. You should apply now. I think hopefully we'll see. Okay. All right. So before we do this 11 to add, one to change, one to destroy. Let us double check. Okay. So that is consistent. We at least only change the random password. Everything else gets updated. So and created. So good. Okay. This is very promising. So let's fix this first and then we can apply our changes to the module. You know, did you ever think that you'd be refactoring? No. Some AI slop line. — Yeah. — I mean, that is kind of what I do like when I do like uh vibe coding or stuff with but usually it's when I'm doing like front-end development where it's like, "Oh, here's all this stuff. " And I'm like, "This is not how I wanted it. " And then I do a bunch of refactoring. I'm like, "Okay, starting from this, let's move forward. " So, it's a lot of like much smaller refactorings along the way. — Yeah. Exactly. And I think that's a trick though. I mean, hopefully when you're looking at this folks and you're looking, if anybody who's looking at the playback of the stream, our intention is not to show you the perfect way to do things. Um, and everybody had been asking like when are you going to do stream on AI? It's not going to be exclusively on AI per se, but if you are using this to enhance your approach, there are some useful use cases for it. um even me who's probably more on the skeptical side of these things um can admit like this is actually pretty helpful. I don't have to do the manual pattern matching myself. Um but you do have to be very diligent, right? Uh you are a human in the loop and yes, you will make a mistake but it's better that you look at it very critically. Whatever output's coming from the um agent, you have to look at it a little bit more critically um before you do it. And in terms of the refactoring practice um you know we're showing you okay extract these resources put them in a module and then apply um in this process even during the apply review that with a very critical eye. Uh don't expect uh these resources to be created or deleted. Right? If you see a create or delete that's immediately something you should go back to your refactoring um your refactoring task and look more critically at. um you ideally want a place of no change, right? So in order to achieve that, there are some things that we're going to do additional. But again, as we're doing refactoring, you want zero, you want your plan to show like zero changes in the ideal. There are some situations in refactoring in which you can do something that's more blue green and then you will have a creation, a create or a delete um task in your apply, but those should be rare, right? And if possible, you want to make sure there's no changes, then you can go back and make changes and like, you know, do something that's a little bit more fancy as it applies. Uh, okay. I guess what version of Terraform are you running? Out of curiosity. 113 this — probably yes I you probably don't want to do it right there but at least we could show queries because I know someone did ask query lists I'll post the link to it for anybody who's interested exactly what we're talking about. Um, so in I think Terraform 1. 14 there was a uh there was a feature added to do import bulk import. Oh my goodness. Um, and so bulk import

is available uh you know you didn't at first you had to search for resources according to their research resource identities. Now you can query sort of based on labels based on any other attribute that you want. Um, and in that case, right, you would be able to retrieve that parameterize like a sort of a search term, retrieve a list of instances or in our case perhaps API gateways that relate to it. Um, and then we would be able to generate a configuration for importing those. Now, we're not doing an import, we're doing actually a move. Um, so it's a little bit different, but I'm asking Richard to upgrade just to be on the safe side. — Okay. Um, — I'll post it on LinkedIn. — There you go. 15. — Okay, perfect. — All right, let's see if the LinkedIn will let me post. I don't know. Sometimes I do, sometimes it doesn't. — Oh, I can I do LinkedIn when I have it. I have it open. — This doesn't sometimes. Nope. It did not let me post. Sad. Oh, how's your How are your — I don't have it in the other. — It's okay. That's fine. Don't worry about it. It's okay. Uh I tried to post it anyway. Okay. It's still fine. Um I guess at this point we could talk about using moved. So um you could tell Bob to restore um API gateway. tf and output. tf. to — I just tell it like just — yeah bring yeah you could just say bring back or bring back API gateway or and output chats TF I deleted it TF yeah I rever I went back to deleted the changes don't tell someone deleted it because it actually might do something yeah there you Oh, that's so nice. Um, okay. So, — well, I figure he lies to me. I figure it's only fair that, — right? Okay. So, the trick though is that you want to do this in the same context if you can, right? Because it has the context. You'll notice um you know with most coding agents, they will give you the context window. Um and so there's context window readout at the top uh right of that IBM Bob window. So it's like 33k. So that's the current context window. It's easier if you ask for the query to do the revert. So you know if you're in the same wind context window if you open a new task then it'll have to reread everything again and then it just gets confused. So it's just not worth it. Save your tokens everyone. Save your tokens. Um — no spend more Bob coins. — Yeah. Well, you used 1. 3 Bob coins for this. Let's just If you click that 34K, you click that 34K. Okay. So, let's talk about it. Click that. Just click on it. Yeah. So, let's just talk about this for a second. Not that bad in terms of tokens. Uh, not terrible, right? You used a half million tokens, right? About a half million tokens. Um, yeah. So, you know, I don't think there's a way you could optimize it to be better, right? uh at least we're scoping it to one specific file. Uh there are things that you might be I'm learning from this from the AI folks because I was like I don't know understand how to do this. But basically there are some ways that you can make this a little bit easy uh you know less uh you know optimize the token usage for yourself. But if you are doing this everybody and you want to understand how many tokens this kind of operation will take here you go. You use that uh caveman mode people were talking about ago. — Oh, did I? Oh, the caveman mode. — You tell it to talk like a caveman and like uses those tokens, but it's like I don't know how. It reminds me of that skit from the office where Kevin like — starts talking like a caveman and everyone's just more confused because he keeps saying SeaWorld, — right? Yeah. Yeah. You see like — Oh, wait. Oh, no. It's fine. It's fine. Oh, inconsistent final plan. — Is this because I upgraded Terraform while it was running? — Probably. We did this the most inefficient way possible, right? Uh, okay. Wait. Expanding the plan for to new uh produce an invalid new value for secret binary inconsistent values for sensitive attributes. Well, I guess that's because we actually changed the password. Um, you know what? We probably just have to Let's see. Scroll up. Did it actually create Aurora

or did it create Aurora? Oh, it did complete it. Oh, it completed it. Oh, okay. I mean, I guess we could ask Bob to debug this for us, but I mean, would it is it worth it? — Uh, I change the plan action from update to delete. I mean, I guess technically, — so what does this mean? Like I if I'm in my real job, I'm doing this, I get an error saying that uh inconsistent final plan, like what? — Yeah. — Why should I read this instead of just going on with my day? — So, what happened is that it's basically saying because we updated the secret um it has new it has a new value, right? And so it's now complaining that the well the provider is complaining that there we've hit some kind of edge case where since we've changed the secret is uh now inconsistent right with what it expected it to be. So originally when it planned it had one set of secrets and now that it plans again it's like this doesn't make any sense. Um, so that's something that we either we could remove this from state since we technically didn't create it. I don't think so. The way we would debug this is terraform state list and see if we can find this AWS secrets manager secret version. Uh, and it probably isn't in here because it was part of the plan but then didn't get a Oh, yeah. So, secret version. Okay. Um, what we could do is we could target the secret version. There's two ways we could do this. We could target the secret version and delete the secret and basically redo it. Um, because I think that the random password, if you go back to the Aurora gateway, uh, let's see. Is the random password going directly to the database or is it going to secrets manager first secret version? Scroll down. Okay. Yeah. So then it's not. And then the cluster is referencing the secret version I'm assuming. Let me see. No, it is not. Okay. So actually this is fine. So then what we could do is we can do a terraform um upline terraform destroy target destroy-target um we want to both the AWS secrets manager and as secret version. So the reason why we can do this is that some um when you create a database cluster sometimes it will just automatically store it or reference it from secrets manager. Uh we are not doing anything directly referencing the secrets manager or the secret version. So we can target it and destroy it and then just recreate it. So basically forcing the plan. Um I think you have to do dash target again. I don't know if you have to do it one at a time but let's see with how it goes. Um, so basically what we're doing is we're just going to delete and recreate it. — Yep. — 10 to destroy that. That seems hot. — Ah, cuz it's reference. — Oh. — Oh, it's our modules again. No, — poor Bob. And we already like we deleted his task — token favorite token just a back turn API gateway. tf into a back intoback. So like go back to API gateway. tf. We're just going to save our tokens. This is just easier. Uh you could copy it into its own file or just save it into a you could copy it and backup file or whatever and then undo like it's fine. Yeah. and then do the output stat backup and then do get checkout on uh API gateway and outputs. — Oh, okay. Do a get checkout. Okay. — Yeah. So, I would just copy the outputs. tf into a backup. Yeah. And then just do get checkout on the output and the you know the original files. This is what happens when you refactor too quickly. Here you go. You should have just made sure it runs first. — I was just thinking that like when we had Bob like refresh it and I went I was like, "Oh, uh, it's probably gonna have another error and then we're redo this all again. " — WE'RE ONE HOUR INTO THE stream and we're still not reactoring. We'll have to make a new another episode of the continued saga of Richard and Rose refactoring this AI generated monolithic terraform

configuration. — Uh it's still trying to destroy all this stuff. — Did you do the undo? It didn't. It couldn't have committed the module already, right? — Uh so I changed this to backup and then I checked these back out. — Yeah. Click on the TF. See if it uses the module. No, it should not. So strange. Is it because it has a dependency? I don't know. Let's try this one more time. — It's still showing. — I guess it has a dependency somewhere. Uh, nope. I don't believe so. Unless it's using it to like access the database, which is also likely. — Oh, yeah. That's why — that's fine. It can delete and then that's fine. — Since we didn't do that makes sense. Yeah, — I figure I was like, wait a second. They all have like some kind of All right, let's create it again. — Okay. Terraform apply. There we go. Okay. Let's hope this actually works this time. Theoretically, I guess we should One thing that's not properly generated and this is not using ephemerals. Yeah, because ephemerals are new enough that it might not be in the training data. — Yeah. Well, ephemeral so this is the funniest thing I guess that the training because ephemeralss were a recent addition I would say in the past year or so probably the stuff that it is that a lot of these models are trained on do not include ephemerals as a pattern. Um and so what I have to do is I put them in a skill. So I have to basically force it as a rule that like if you want to if you're writing any terraform check if there's an ephemeral first. So that is in um there's an agent skills repository that we have if you want to look at that um for your own reference but um there is a note that basically forces it to look for an ephemeral first. It's still creating it. It's fine. Anyway, uh so while that's happening, I guess we should look at other things to modularize, we did API gateway. Um and so next thing behind the API gateway is lambdas, right? I think let's see lambdas. — Uh probably. — Yeah, I see lambda permissions. Those are EKS. So scroll down. We'll go back to the Lambda. Yeah. So, I think it's probably Lambda functions if I had to take a guess and like the associated Lambda IM identity access management stuff too. — Yeah. So, scroll down. Yeah. So, there's like roles too. Uh, and so this is where, you know, you probably get a little bit of a question on like when you refactor, what is worth modularizing? What's not worth modularizing? Like, uh, I don't see any IM policies that are cross lambda. So it's like in terms of the logical grouping, it seems to be lambda function plus the IM role. Um, and the IM role is declared in a different file, right? Is there anything else here that we should be aware of that's kind of like Oh, it's already scheduled for deletion. H — I I'll got this. Hang on. It's because by default it just schedules a seven-day deletion and — Yes, I know. Yeah, I one day one day. — Uh yeah, give me one second. — Yeah, — you could just rename the secret. I guess it's rename the secret in terraform. It also works too. — You could the secret in terms — and then um Aurora. Yeah, — then I have to like I'm have to do a bigger go find everywhere where it's referenced. Oh, this name. — Oh, yeah. Do that name. — It might be too. — What' you say? The audio. What' you say? — It might be long, but it might be too long. Don't know. You know, this is what happens when I just ran it once and it applied and I was like, great. The only thing that didn't apply was the quantum ledger and

I was like, that's great. So, I took out the quantum ledger because I was like, I don't really think we need that. It's kind of like, you know, anybody who's doing this, it's not really strictly necessary because there's not actually any real code in this. There we go. All right. Now, we can go back to our refactor. So, take the backup and put into the form. And what we were originally doing This is uh trying to delete them again. — That's okay because we don't so we didn't do what we need to actually do this part, right? So our problem is that it's trying to delete the old resources and create the new resources in the module. This is not what we want, right? We ideally want them to be moved and conveniently there is a move block in Terraform. So um the move block is labeled with moved and then you go from an old address to the new address, right? Um makes it kind of easy. The thing that we want to be really careful about when it comes to moved and what's kind of annoying, right? You'll notice that scroll down, Richard, on the summary. There's like 20 to five to add and 25 to destroy. That's a lot of resources. Um, you know, I guess there are like two ways that you could do this. Uh, there's like two easier ways I guess you could do this, which is what kind of uh someone in chat was talking about. You could remove uh some of these resources from state. Um you still unfortunately need the address which is irritating uh and then you could import them back into the module, right? So you could remove the one the old address out of state and then import them into the new module. That actually still doesn't solve the problem because you still have to find the old addresses, right? Um, and so this is where it's kind of nice to have an AI agent to help you with this. Uh, so the first thing we're going to do is the apply and then we're going to export it to a JSON. So you can do Terraform apply uh, Terraform plan. Sorry, not — Yeah. Okay. How do you do it plan? — JSON plan JSON. I think you could just do JSON now because I think you upgraded, right? So, I think you could just do that. — Yeah. — I don't remember. — I just do that — on JSON. Let's see. — Uh oh. — I got to pipe that somewhere. — No, I mean I think it Hold on. I think we actually have to do this that plan again. There we go. Okay, whatever. That's the logs, which doesn't really do anything for us, I guess. Did they actually pipe it out or no? Oh, some plan. No, it didn't do anything. Okay, we're going to have to do the show JSON again. So, uh, Terraform plan dash out. And then you already did that. And so then what you're going to do is — like 100 kilobytes. — I Yeah, because it piped it to a binary which did nothing. Oh. Um, — yeah, that didn't do anything. Uh, yeah. So, we're going to do terraform plan and then dash out TF plan. Yeah, TF plan is easier. Whatever. Do it the old way. And then, yeah, hit enter. You don't even have to do the out. It's fine. It's just a file, binary file. It's its own Terraform has its own binary file. And then, yeah, I will just do all that. And then terraform show space JSON terraform space show and then uh show JSON dash JSON show dash JSON and then the file name TF Yeah. Yep. — That tab I'm planning for. — Okay. So then you can write that to

Oh, this Yeah, you could write that to wherever. Yeah. TF plan. json. I think it's actually the same thing. We could have just piped it, but it didn't pipe out there. Anyway, so if you do TF plan JSON, that should have it all. Okay. So now we'll get really really specific. Okay. So now we tell Bob, uh look at TF plan. json. json. Um uh find the addresses of resources to be deleted. uh and uh created. Okay. And then add them to a Terraform moved block. Okay. So then um you probably want to give them the example. I don't know how you want me to uh Is it better through Slack or private chat? — Either one's fine. I have it open in the uh a different screen. — Okay, let me do that. — I have an I have like I have — the move example. Yeah. So, it would be Yeah. Yeah. Exactly. AWS. Uh Yeah. So then you would say like AWS uh you would say resources you the from because it's not that just to be very clear from res address of AWS underscore whatever API gateway v2 API whatever — one of them choose one of them because if you're not specific one thing that I've learned is they love examples if you don't give it a specific example it does not know what to do so better to give it an example of what you are looking for. Yeah. So like that API gateway stage config that yeah put that in there and then it would be moved to module whatever it is API gateway stage do this or something like that. uh dot stage config I don't know stage config I think it's under config right module dot it's got to be config whatever is the config yeah moduleconfig and then stage this yeah so you want that one okay there we go Okay. And then let's see what this does. No, I don't want to do this. Start new task. I guess I don't know. Why does it say the context window? — is TF plan too big? There's no way this is too big. — It says it's 823 kilobytes. — Is that too? Well, it's not the kilobytes. It's how many tokens are in there? — Wait, Yeah, — that's 200k tokens. That's not — That can't be 200k tokens. — Too much context contesting are start a new task like trying to reach LA file. You could use the large file read strategy in chat settings to help avoid this. This is why you know what? We should have just written a Python script. — I wonder — you want to just pipe that out. Maybe if we pipe that out. — Yeah, that that's what I was going to try to do. — That might be more token friendly, which I hate to say. — This is one thing everybody, you know, we had to apply our whole I remember we talked about whether or not we could do a condensed version, right? It was a compact version or you were thinking of like — a compact version, but I don't think there's a compact version. — Oh, this — sadness. — It's because of the — Oh, no. Yeah. I knew there was a flag. I mean, I guess we could do said and just, you know, a grip. The old good oldfashioned grip. Anybody have any ideas on this one?

We could just grab. We should just grip. Actually, like I don't know why we're just grabbing. Let's see. Maybe we'll yell at us. the context window. See, this is one thing that folks, you know, you have to do this by hand manually, and I don't like doing it by hand manually. So, previously I would write a Python script to go searching for this in the JSON. Um, I'm not sure that I got it that this is better, but you know, hey, experimenting. — Why? Why are you cute? Unq probably because the API thing. I don't know. Let's see. I don't know why it's cued. Maybe because the API is pooped out. — I used all my coins. — Did you? I mean, there's no way. — No, — there's no way you used all the coins. Maybe copy that and try again. Or reload. Control. Oh, nope. You're just gonna That works, too. Okay. — Yeah. The original reload. — I was there. It's probably going to hit the context one. Oh, no. Okay. All right. Okay. So, note to self, don't use JSON. — Oh, I don't Oh, I hate this because it reads it in chunks. It only reads a hundred lines at a time. But the problem is it's like all the creates and then all of the deletes. — Yeah. — Oh. Oh, — I mean, okay. So, if you really wanted to be smart about this, right? Okay. Well, I guess that kind of works. If we really wanted to be smart about this, I guess we would have like just grepped like in our processing would have gpped state refresh because that those are initially the old resources, right? Okay. So, let's see. Uh, did this work? I You're Oh, you're just going to yolo it. — Well, I did a plan. No, I know. But I was gonna actually look and see and make sure like it actually did the move. — Bob told me run this command and I do what Bob says. — Okay. All right. Let's see. — It says zero to change, zero to destroy. — Okay. All right. So, I guess that was easier. I didn't like have to write a Python script for it. And I — Yeah, look it up grip commands. — Look up grip commands. So, I guess it's kind of easier. Um, yeah. So, it does look like it has uh officially thought about how it's going to move everything, which is good to a certain degree. So, I think that's consistent. We can apply and then see how that goes. Okay. So, we finally got the moved. So, we moved 25 resources after doing a refactor. So that was great. Uh, and that's good, right? We want this to preserve in state. So we just want to make sure we apply it. And then after it's done, you delete the moved, right? That's literally all this is why this is kind of like the most perfect uh opportunity to use some kind of use case uh some AI use case for this because quite frankly like this is something very temporary, right? Why would you use a move block versus using a CLI? I mean you could script a CLI. there was a CL CLI capability like if you did an import remove and import you could all you could do it through all through scripting if you really wanted to but move makes it a little bit easier because then you would like if you have a real infrastructure as code setup you would push this declaratively to a um to your um version control and then your CI pipeline would apply it uh and then it would just pretty much preserve state and then you could just remove it on the next commit. Okay, so we've done this. Um I do want to check something. Can you click on the tokens? I want to know how many tokens this is because well, we're being I'm being very scientific because I'm very curious. So, interesting enough with the TF plan. ext. So, the trick is you output the Terraform plan to TF plan. ext. Um, and then interestingly enough, only took about 134K 130 about 150K, let's say, in tokens, which is interesting. That's very interesting to me. Um, so that's for like 25 resources that we moved. Not a bad, not a bad thing. I don't know your thoughts, Richard. Would you do this again? — Uh, I would do this again. I think it just proves even AI hates JSON. — That is true. Yes, I suppose it does. Okay. Uh, and so that's one refactoring. That's one way that you would refactor. Um, and so then we could continue refactoring. I don't know if we want to or if there are other things that we want to do instead. I guess we could

show query lists. Um those are for our next one because we do have to refactor the lambda. So we could refactor the lambda. — Oh yeah, that's what we had started doing and then we uh — Yeah. Right. — Now the lambda is a little trickier because the lambda actually has a file that it references. — Well, yeah. This is what I was confused because like so when it — there's not actually anything in there. — Yeah. Well, I had I unzipped it while something was running and it's just like it just echoes back a 200 response code that says placeholder. Um, but it's like they're all referencing the same one. Like it's just like an artifact of like the AI being like, "Oh, here's a generic thing for you. " They might be like, "Oh, every lambda you create is going to use placeholder. zip uh and like try to make that like a hardcoded thing. " — So, let's see. Let's see. We'll tell it. So um the thing is that this is why we don't want to tell Bob uh I mean you could ask Bob to give you a refactoring plan. We're not going to do that. So we want it to refactor um IM serverless as well. So Lambda and IM serverless both of them. That's because each role has been configured here and correlates to a lambda. So we might as well create one module with a lambda plus an IM role. I don't know unless you think we should do it separately, Richard. But I feel like — No, like I think that's what you would pass into a Lambda. Yeah. Uh like configuration is like here is like you basically say like here's the code I want it to run and here's the role it needs and that is a appropriate abstraction, — right? Uh so yeah, we can get that started. We can ask Bob to help us. But I am serverless and lambda. tf TF uh refactor into a module. Um so there's something again there's something to be said about how do you identify a logical grouping during refactor. Um sometimes things just belong together. They don't have any dependencies on anything else. So like if the IM role was being used for something else then maybe I wouldn't be grouping it together with the lambda. But since it's a onetoone mapping then I'm going to group them together. You can't really create a lambda without an IM role anyway. So, you know, um those two dependencies get kind of looped together. Um that's logical, right? There are some other things that might not be so logical. Uh and so you have to be a little bit more careful about how you approach the refactoring effort. Uh and then there's also an argument to be made that sometimes you don't want to necessarily refactor into a module. So, we're doing the API gateway, we're doing the lambdas because those are repetitive, right? Don't repeat yourself as a principle, as a coding principle. Um whether or not you follow that sort of up to you, but like If you don't want to repeat yourself, you have four Lambda functions and four API gateways. Those are things that we could probably put in modules and then reuse the configuration for those. But let's say the Kinesis stream, I think we only have one Kinesis stream, one Cognto instance, things like that. And one Aurora database, those things we probably don't want to put in a module. We don't really need to. There's not a necessity to do it. Okay, let's see what message what mess it created. — He's still working. — He's still working on it. Yeah. And if anybody is following this and you're like, "Oh, I want to use my own um you know, coding agent, whatever. " Uh you are more than welcome to. Like I said, your mileage may vary with the prompts that you're seeing here as well as the approach that you're seeing here. Again, we're not using any special skills. So, you know, you could apply your own skills and make this process a lot easier. — Leave a comment in the chat for how well it handled the JSON. — Yeah, I know. I want to know. Did other ones handle the JSON the plan JSON? Okay, so it did something. Um, okay. It refactored five Lambda resources. Uh, and it only kept the Fargate and API gateway roles. Okay, that's fair. I That's good. It didn't try to refactor everything. That's good. — Um, excuse me. — Yeah. So, Lambda, I guess this is where we're going to look at stuff. And it forgot the Terraform file again. I swear I I'm about I'm like this close to saying we just need to shove a skill, a local skill until it trade this terraform file because this is ridiculous. — Oh, actually it takes the file name. I was worried it would like — think that the file name is like oh it'll always be named this. — Thank goodness. Um — on the other hand, it's made everything else like

— you have a module that like takes a bunch of variables that are like uh — all the variables are passed like directly through it's like well how much abstraction is the module uh abstracting if like everything is surfaced but at least — I like to be more opinionated personally. So like if our runtime is never going to change. Did it default to runtime? I hope it I like let's hope it actually defaulted to runtime. Um Yeah. So like — nice — if you in your organization you never ever ever ever have any other runtime other than Python 3. 12 then maybe you just hard it could be hardcoded right like fine just do it that way right um VPC config I mean like from a security standpoint if I always want to deploy to a VPC I'll like just basically create the module with a default VPC config I wouldn't bother doing the dynamic block that it's done here but you know again it's a it's ultimately sort of a design decision ision that you have. If you know that this is you're very strongly opinionated on um certain configurations like you're always going to attach this lambda to a VPC then you should just do that right — this seems like one of those things that's like — yeah partition should also just be hardcoded I mean not some people do like gov cloud and stuff like that but — yeah okay good enough u now again if we run this again it will tell us that it's going to create things and we don't want that. Uh, so we're going to do Yeah, just you could do plan new plan. ext. Oh, got to do terform a minute, right? Forgot about that. Okay. What did it do? — Um, it's undeclared race. — Ah so it because we scope the context. — Yeah just to that um we can you can actually tell uh Bob in the same context uh say terraform plan comes back with errors uh update what is it s3. tf to use the modules reference and then there's uh s3. tf TF and then anything else that lists has a list cannot be found. Yeah, cannot be found. — Uh if you scroll up, the other thing we just want to double check API gateway as well is affected. And what are the other files that have been affected? If you scroll up, right, — API gateway, API gateway. — Yeah. So, you also want it to tell it to fix API gateway and S3. Oh, there we go. It's automatically doing it. Nice. There we are. — Oh. Oh, it's searching for Oh, — yeah. It's very It's clever. It like searches the directories. Okay. So, I think it's fixed it. Okay. So, before we do this, um, we can see if we can try doing query terraform query. Um, I think with this setup it might not be so fast, but you know, it's okay. Uh, all right. So, the question is we need to be able to find the type and the label. And then the label is uh kind of the tricky part. We have to figure out if there's like a common tag. So, let's see. Is there a common tag with these lambdas? Uh I think so. I thought I saw it like add like a um — oh bar tags. What's a default tag? See this is the problem with the query tag query. It's assuming that you're very disciplined about the tags. Um because it is basically selecting based on some kind of attribute, right? So the question is which attribute and how? Although you could just like say I just want everything which is okay. I guess we could do that too. Although I'm pretty sure Richard you have a lot of lambdas deployed. — Oh in my account. Oh yeah. Oh like a dozen. — Okay. Well what we're going to do is we're going to ask um we're gonna uh because you put the new plan. ext um if you could do it again. — Oh. — Oh yeah. No color. There you go. Okay. So, how many of this is this? Is it going to change? That's my question. How many resources are we looking at here? We're looking at 23. — What are five to change? Oh, because that's the API. That's the the the

end point. Yeah, because it's assuming that it will create the modules and stuff like that and whatever. So, uh this is not completely correct. So what we want to do but first if you want to do query in this way which I don't recommend I think you should just use move use your code happy coding agent to go through and do the moved conversion for you that was pretty fast um but we're going to try using terform query so uh the first thing we're going to do is we're going to use the removed block to take the deleted the um deleted resources. So you'll notice in that list there's like 19 to destroy. Um so we're going to tell Bob to use the uh create the removed block instead. So we're going to say like in new plan start new task because we don't want it to add to the context window. It will be too much pain. Okay. So at new plan. ext um create removed blocks for the 19 resources to be destroyed. It's removed with a D. We'll see if it's smart enough to know what exactly search this direct. Oh — yeah, it's so what we're going to do is we're going to pretend that these resources no longer exist in Terraform states. This is a bit dangerous, right? Because if you don't remember these the addresses of these resources and the ids of these resources, you will basically do something where you're kind of like well you're shooting yourself in the foot on this one, right? To use that metaphor, uh it's like basically you know you're basically saying all right, these are gone from Terraform state. they're not managed anymore. So, you're doing a removed and then an import. Do not do this in your general refactoring workflow unless you absolutely need to for, you know, some other uh strange complicated use case. But, we're trying to do Terraform um query just because a lot of folks, you know, are interested in seeing how it works. So, we're going to do a list uh we're going to in a file in a Terraform file create a new Terraform file and you can do list. tf or something or query. tf. I think it's called query. Hold on. It's called query. No, it's not. It's um TFquery. hcl. — That's the name of the file — you could do. Yeah. So the name of the file we could call like lambdas tfquery. htcl. Yeah, there you go. Okay. So then we're going to do a uh Yeah. So we're going to do a list block and then you can do AWS Lambda. I think that's the ADS Lambda function or something, right? Yeah. AWS Lambda function. Yep. that thing list uh and it will be quote in quotes next to the list kind of like a resource and then the AWS Lambda function and then the ID. It's the same um syntax. So, it's list and then quotes AWS uh lambda function and then you could call it like I don't know uh monolith I don't something yeah deleted lambdas I don't know they're not actually deleted by the way folks they're not these resources are not actually deleted they've just been removed from Terraform state and I'll show you why that looks like that uh and then uh provider is required so it's provider equals and then in quotes AWS or not in quotes. Not in quotes. Sorry, not in quotes. Um, how many lambdas do you think you have in this account? — Oh, this uh — we'll just do 50. Limit equals 50. Um, yeah. Okay. So, I think that should be okay. I don't know if we need to filter anything, but we'll see. Uh, okay. So, we have that. And now that we query, we do Terraform query. Let's see. Uh, now it will get us all of the lambdas that exist in the Terraform in the account that Terraform is connecting to. Okay. So it tells you the account, a bunch of functions

and the good news is it does tell us like, you know, like there's, you know, every single lambda that's in uh the account. So that's good. I guess the problem is how do we get these out? — I have Yeah, I haven't tagged any of them. — You know what we do? Terraform query space uh dash generate config out generate-config equals generated. tf. and all of one generate config out is all one. So what this will do is equal to generated. tf. Okay, so this doesn't actually do what we want it to do for better for worse. Okay, folks. Um but if you had the situation where you needed to import it back in, Terraform can generate you like a file uh generator. tf TF that has the Lambda functions all created back in there with all the hard-coded things, right? Um, and it will import them for you. This is not what we want, right? We want to import back to a module. So, for better, for worse, this doesn't really help us, right? Because here we are. Um, — I could probably use like the AWS CLI to list my functions by the last updated. And if I set it to like the last two days, it'll like I've only I haven't touched any of this other stuff in that account. So I can just get like the list of function names if that helps. — Yeah. I mean otherwise you still have new plan, right? — Say it again. I still would. — You still have new plan. ext. — Yes. — So click on new plan. ext. Um there if you scroll up, go to the ones that they say are going to be deleted because we didn't we kept this plan. Um scroll down. Yeah, keep scrolling to the actual like lambda function. config. Yep. Okay. So, scroll down. Let's see. Let's see. Okay. So, it has the arms, right? And that's the important thing, right? The Yeah. So, it has the arms. So, what we want to do is uh we already did the remove, right? So if you do a terraform plan now that Richard um we removed we have a removed file that basically removed removes the lambdas from state. So you could do a plan uh what happened now? Oh remove generated. tf. We don't need that. Yeah that was not what we were looking for. So, okay. So, we shouldn't see the 19 destroy. Oh, do we remove removed resources? Not — I didn't apply that. — Like I — No, it should just tell you though in plan it should be because it should be if you look at removed, right? Removed resources. It should just not Hold on. Why is it Uh, it doesn't manage it, but life cycle should not do anything. Oh, well, it destroys the actual resource. So, destroy false. Yeah, everything should be destroy false. Uh, no, no, sorry. In the removed resources. tf, you don't want to destroy them. You just want to remove them from state. So everything destroy true turn into destroy false. Yeah, we're doing this the os everybody just use moved. If you're doing refactoring just use moved. Don't do this. But you know we're doing this anyway for the sake of the exercise because moved is basically a state removal and then an import right into a different part of the state. So that's really what it is. It's just moving it around. Um okay. So it will discard the tracking information but not delete them. So that's okay. You can just do terraform apply. So now you have uh Oh, don't apply. Sorry, don't apply. Just keep it the way it is. Keep it. Don't apply. Uh, okay. So now we've kept the old new the new plan. ext. We kept the plan that had all the IDs. So now what we're going to do is ask Bob to create a bunch of import blocks importing the uh Lambda resources and their IM roles into state. So you can say at new plan. ext text. Um create import blocks. Um importing the lambda and IM roles

into the module state. Let's see if it interprets that. That's very vague. I feel like we could be more specific, but whatever. Okay. So, we're telling it based on this plan, uh, use the import blocks or create the import blocks for me. Okay. Now, the question is, will it do it? Uh, so does this is this correct? ID. Now, my question is, is this ID actually the case? Hold on. Lambda AWS Lambda function. This is something where we got to do the resource. Uh so ID, what does the ID? It's not an ARM, right? That's the — Well, we could also use the identity attribute or we use the ID. The ID is the function name. So yeah, there we go. Okay, so that's not bad. Yeah, the ID is the function name. So the question is, did this actually do what we want it to do? Um, if you do a Terraform plan, we'll see if it does actually what we want it to do because these lambdas exist. We theoretically remove them from state. If you scroll up, 19 to import, 18 to add, five to destroy. Oh my goodness gracious. What happened now? Um, why does it think that it is going to I guess the package this is why we have to be very careful. Uh so the attachment was recreated must be replaced imported from okay so it's changing the name of the role is not lambda roll — oh it's it probably did something oh no far it kept Um policy. Wait, where are the ones that it imported? — It would have done it as part of the import is under import, right? — Oh, I meant I'm sorry. Uh — oh, — but it creates the lambda functions and it passes in a world use. Where is that? — Oh, the module might override it because it's not using lambda ro, right? It's just appending role. So if you go to the module and look at the IM roll for the lambda. So yeah, so it's not actually doing Yeah. So we probably want to do lambda dash rule and then preserve it. So that's the one thing again being careful about refactor when you parametize stuff like this like just be conscientious. Uh like it this is okay, right? I would say this is sort of like a trivial change of just like renaming things. But that's one thing I don't like about uh sort of renaming everything in bulk. Um it's better to just preserve what you can and then you can always rename things and make it a little bit better. Uh but this is one of those situations where you want to catch if that change is something you really want or you don't want, right? Um this is very trivial. Like I said, this isn't this naming thing in our case doesn't impact much, but I personally want to preserve as much as possible. I don't want to mess with too much. So the minute it tells me that there's a change, that's when I get very cautious and I go looking again. So that's kind of like, okay, what else is it telling me that it's going to change and destroy? Um, so the attachment is interesting. What is the attachment for and why is it creating it versus anything else that it didn't have — creating this new — policy? There's also policies too. I think the policies were probably uh also done differently. So I think the attachment probably got a little uh conditional attachment. I mean also that ends well right in your mind. You're like this is fine. I guess lambdas are fairly as a resource they're fairly ephemeral to begin with. So perhaps these things aren't terribly disruptive. But again due diligence because some resources are not so ephemeral and you want to be careful when you make changes.

Uh — yeah, — those things. I mean, I think this is okay. I'm okay with applying it. I mean, I don't I'm just more cautious about what needs to be destroyed. So, if we can find the ones that need to be destroyed, why does this say this roll has to be replaced? Is it because the name? Oh, the name. This is very ridiculous. Okay, let's go find the policy again. And I am there's got to be policy created somewhere. Okay. Yeah, put lambda in front of it. I just like I said these things policy not so bad. Policy creation is very fast. Ro creation very fast. But again, just for like my personal due diligence, I just make sure to preserve as much as possible. So nothing is destroyed. That's good. Um 19 to import, four to add. Poor. — Should we say yolo and just run it? — Sure. — Because we have 15 minutes left. All right. We're just gonna say how see how it goes. Everyone wish us luck. Everybody's like, "Why are you doing this? " Here we go. — I am curious how well it works because I know like there's this weird like circular dependency thing you need to break with like S3 where it's like if you have a Lambda function that's triggered by an S3 event, like someone putting something into an S3 bucket. Yeah. that you need like the bucket needs permission to invoke the lambda function but then the lambda function needs permission to like read from the bucket because it just gets a notification. So it's not clear like you have to like hardcode one of the things so it doesn't create this like cycle in your dependency graph because the function needs permission to invoke the lambda and the lambda needs permission to read the bucket but like — you have to start somewhere. — Yeah. Exactly. Okay. Yeah. I mean at the end of the day you know it's at your discretion right? I try to preserve as much as possible, but some resources are very ephemeral. They the changes apply very quickly. Like mutable changes apply, immutable updates apply very quickly. So like a role and a policy, the changes apply fairly quickly. As long as you're not changing the policy itself, you're just maybe changing the name, then it might not be terrible depending on whether or not you have upstream dependencies that require access to or whether or not you have downstream dependencies that have access to the role for some reason or the policy. I'm not sure. Again, at your discretion, um I don't think in terms of the blast radius that we had looked at for this lambda and this role, it wasn't that bad. Not that big. So, yeah, changing the name isn't so bad, — but All right. So, here we are. Uh, well, we've applied things. Not much has changed. I think most of the stuff is pretty much the same. Um, — and then after we refactor, we clean up our clean up the stuff we left behind. So, that means remove resources, new plan. ext. — Um, yeah, all those things. Uh, you know, is it what else? New plan, anything else that we like kind of left behind. Uh, graph — graph. I We could keep the graph because I guess does do we want does anybody want more of this kind of stream? Do we want to do another episode of this? Have we left this the way it is? I mean, we technically didn't refactor everything, right? So, we're not really done. There's a lot more to refactor. Um, but at least you all got sort of a sense of like the two patterns that you can take. So, I suggest prioritize moved blocks. um use your helpful AI coding agent to do it. Get the plan out into a text file. Have the agent generate the move blocks for you. Right? More straightforward. Um and you could do that in bulk. If you find that you have resources that you've accidentally deleted in the process and you need to import them back in, um go find the IDs for those resources and use the import blocks to import them back in. We also use TFQ to kind of generate those import blocks. uh didn't really suit our purposes because we're using modules. So, doesn't really work there. Um but it may be helpful for you to just do like import and then do a move, right? So, there's a lot of options there if you accidentally did something and you need to go back and do it. Um so, you know, just be aware. But, uh we also showed how we use the graph dependency graph to chart out and look roughly look at the blast radius of various components. So we started from the rightmost part of the graph where there are fewer dependencies and we're working our way back to sort of the more challenging complicated dependencies. Um and in every step of the process we don't want changes, right? Changes are a red flag to us. We want to go back

and we want to look at those changes and assess whether or not they are something that we should be concerned about or if they're just sort of natural part of refactoring. — Yeah. Change is bad and should be avoided at all costs. — Yes. Exactly. Yes. Change is bad in this case. in this case, change your infrastructure every other time, just not during refactoring. Uh, Richard, do you have any learnings? Any thoughts? — Uh, I mean, I learned more about like tricks to get Bob to do something very specific. So, I'm usually just like, "No, Bob, stop doing that. " Uh, like narrowing his context to specific files. Yeah. Um, and then the uh using the import blocks. I had done those in like a very like trivial things, but nothing like as big as what we did. What was like 25 resources we imported all at once was uh exciting. — Exciting. Nerve, I'm not sure. — Eventful. Um, — yes. I don't know if this is worth one more episode if just for the sake of like completion, but also I think we might have we might encounter more trouble like the more complicated components. But — yeah, — you know, I think it's worth um maybe we'll do another episode next week where we we'll hopefully finish off the rest of this refactor. Um but in the meantime, again, if you missed previous episodes, you want to review this one again, uh you know, we do have our previous episodes in our playlist. Uh if you have questions or comments or if you want to see some other refactoring before the next one, let us know what kind you want to see and maybe we can make it happen. Um otherwise uh enjoy. Good luck with all of your infrastructure as code or not infrastructure code depending on whatever you use and we hope that you'll join us for the next episode. — Cool. was paint.