Your Kubernetes cluster is not secure. Secure it with Kubescape!

your kubernetes cluster is insecure no like really insecure take a look at this list of vulnerabilities that I found in my cluster hundreds of compliance and configuration issues as well as vulnerabilities in the containers that I'm running now the good news is we have a way of finding and detecting these issues in our kubernetes cluster using a tool known as cubescape now if you don't know what cubescape is basically it's a free to use open source tool that you can use to run scans in your cluster but not only that you can actually use it to run scans on your code repositories and your image Registries and this is really important and you're in the devops security or just kubernetes realm in general so in this video I'm going to go over everything you need to know to get started with the tool cubescape and I'm also going to go over the armo platform which is the SAS solution that cubescape can integrate with now this video is quite long it's actually quite a lot longer than I intended but that's because I go over a lot of the cool features that I really like with cubescape as well as the armo platform so I'm going to add chapters to this video and just use the chapters to find the sections that you're interested in I go over everything from using the cubescape CLI how you can use it to scan your kubernetes cluster scan your yaml files as well as integrating it with GitHub actions to automatically scan your manifest anytime you make a change to them I also go over how you can set up registry scanning as well as visualize everything using the armo platform and a couple other features that you can do in the armo platform as well one quick disclosure is this video is sponsored by the armo team their product is free to use if you're running a cluster that has 10 or less nodes so everything I did in this video was completely free but just wanted to get it out there that this is a paid video and I wanted to thank them for sponsoring a Creator like myself now the best way to actually learn cubescape is to see it in action so let's go ahead and jump into the video all right so let's go ahead and get started with the cubescape CLI so if you haven't already you can install the cubescape CLI using the custom installation script or you can use Homebrew I'll make sure to give a link in the description below for instructions on how to get it installed as well as some of these commands that I'm running here so I'm going to run the First Command which is just the cubescape scan command and basically what this is going to do is it's going to reach out to my kubernetes cluster and it's going to scan it for vulnerabilities and compliance issues and you can see that it returns a very long list of compliance issues and vulnerabilities on my kubernetes cluster some of these scores are looking pretty good but a lot of them are pretty low but to go over this a little further it has the severity it has the control name and how many of our resources failed this actual compliance issue and then it gives us a score so we're going to learn a lot about this as we go on with the video one thing I want to mention here is when you just run the scan command it runs it against all the different compliance framework so there's a lot of different compliance Frameworks out there by default they give you access to the NSA compliance framework as well as the Mitre framework now if you want to run a scan specific to a certain framework all you need to do is Rerun that scan command but then you can specify the actual framework so I'm going to specify the NSA framework and you can see that the list that it returns is a lot smaller than the one we had previously and this is because it's only checking for compliance issues and vulnerabilities that are part of the NSA framework we also have the ability to create our own Frameworks and that's going to be something that I show you later in the video and we can actually create our own controls as well them using a GPT enabled tool so I'll show you how to do that later on in the video now to go back to this we can see that we are getting a resource summary and our combined compliance score is 58. 30 percent so I'm getting flashbacks to my high school report card this is a pretty low score now one thing you can do with this compliance score as well as with the severities is you can make your CI CD pipelines actually fail if a certain severity comes up or if a compliance score isn't reached so we could set a threshold for say around 80 percent and if our compliance score was lower than 80 percent or had any critical

vulnerabilities cubescape could actually fail our pipeline so that's one thing that keepscape can be used for that'll show as well later on in the video I'll show you how to set this up in GitHub actions to automatically run cubescape as a GitHub action whenever you can code to repository but before we get into the actual pipelines I want to show off a few more cubescape scan commands so I'm going to go into my other window here and in this window you can see I have a deployment. yaml file so one thing I can do is I can actually scan all the yaml files in my directory so I can go cubescape scan and then I can do a wild card so I'll just do star Dot yaml and basically what this is going to do is it's going to read all the ammo files in my directory and it's going to scan them for compliance issues so you can see here that we have 65 controls in place for our actual deployment yaml file and 11 of them failed and 50 of them pass and it gives you a summary of what those controls are so having a look here we have a high severity for resource limits and if you know what a resource limit is in kubernetes basically it's a way of limiting the amount of memory and CPU your pod is going to consume so this is a really important parameter that you're going to want to put on all your deployments so it makes sense that this is a control that cubescape can look for having a look through a couple of other these ones you can see that they're just sort of your general best practices not running root containers not allowing privileged escalation some Linux hardening so these are just various things that you may want to fix so that's how you can look at yaml files in your local directory let's have a look at how you can scan an actual repository on something like GitHub so I'm going to go into another window here and I have a command ready and it's our cubescape scan commands but this time I'm specifying one of my GitHub directories and I'm setting an additional flag here for compliance threshold and basically what this says is I want my compliance threshold to be at least 90 percent otherwise return back false and this is one of the ways that you could integrate cubescape into your CI CD pipeline to actually fail the pipeline if the compliance score isn't met so you can see at the very bottom since our compliance score is 76. 92 percent it returns back a failed state so if this was a pipeline it would make it fail all right so we had a quick look at cubescape from the CLI perspective let's now have a look at how cubescape can integrate with the armo platform all right so we are currently in the armo platform right now and as you can see I have two clusters set up and it's actually really quite simple to get your clusters connected to the armo platform you can do it directly with the armo CLI and just specifying your account number or you can actually install the cubescape operator into your kubernetes cluster using a Helm chart and that's the way I did it I'll go ahead and show you how you can do that as well if you're logging in for the first time I believe it's just going to give you a pop-up of the how to install the helm chart and you just copy and paste it and that's all you really need to do to get cubescape installed into your cluster so it's really simple to get it set up let's go over the left hand side all the different features that are currently available on the armo platform there's going to be more in the future they're constantly developing new features but what's available right now is this simple dashboard view which gives you the configuration risk and vulnerability risk over time so it's a nice high level overview we then have the compliance section so with the compliance section this is how compliant your clusters actually are so I talked about those different Frameworks like the National Security Association framework and this is a standard framework that the National Security Association gave out on how to secure kubernetes clusters so it's a good one to follow but a lot of those requirements are actually pretty hard to meet for normal day-to-day use kubernetes clusters so one thing I like to do is create my own Frameworks and I'll show you how you can do that the next thing we have is vulnerabilities and this is going to be all the different vulnerabilities inside your cluster so this is things like the actual images that your pods are running so if there's any vulnerabilities on those images that the pods are running it's going to show up here after that we

have the rbac visualizer this is just a way to visualize all the different accounts that you have in kubernetes and sort of give a high level overview of how everything is connected together so we'll go through that at the bottom here we have repository scanning and registry scanning and these are actually really great features that you should be implementing into your security stack this is going to actively scan your GitHub repositories as well as any image Registries that you want to scan as well now the last thing is the settings at the bottom and this is where you can actually configure or remove clusters set up your own different Frameworks controls and integrate with third-party applications like slack or Jenkins so a lot of different things underneath the settings later on in the video I'll go over the most important ones so before we get there let's head on over to the compliance section and here you can see on my two different clusters so I have the k3s one as well as the production cluster that I was talking about in Amazon eks and it gives us a summary report here of all the different failures so if I want to go into a specific cluster I can just click one of these here and I now have a list of all the compliance issues now this list is very similar to what we were looking at before when we were using the cubescape CLI to actually scan our cluster but what the armo platform is doing is it's giving us better visualization it's recording the data and it allows us to sort through the data and actually work with these issues on a platform so if we go ahead and look at any of these different things I can hop into this first High severity one where it's saying app application credentials in the configuration file and you can see out of the 88 resources on my cluster four are failing this check and one of them is already ignored so if I want to actually action this what I can do is I can hit this fix button and what I'll be presented with is the different objects that are not compliant with this control so you can see that I have a couple configuration maps and two jobs that are violating this control so one thing you can do is if you want and you know this is not an issue is you can actually ignore this check so the next time you run the scan it will no longer trigger it so if I want to ignore it for this Argo CD configuration map I can hit ignore and I can enter in the reason I can set an expiration date on the ignore or if I just leave it as nothing then in this ignore will never expire I also have the ability to change the scope of this so right now it's scoped down to a specific cluster and a specific namespace on a specific resource but what I could do is I could actually scope this against all my clusters so if I wanted to change it to be a part of all my clusters I could do that right here and that way it's going to ignore this specific configuration violation on both my prod cluster as well as my lab cluster you can do the same with the namespace as well as the actual resource so I'm going to go ahead and hit save on the ignore here now the other option that you have instead of ignoring the problem like I do with most of my problems is you can actually fix the problem and the way you do that is you click this fix button and it's going to bring you into a new window now to view this I'm actually going to go back a level and I have a more interesting problem for us to actually fix so let's go ahead and add a different framework here right now I'm just honed in on this specific framework uh but I want to go into this NSA framework and show the violations on this one so I'll just click on this and now I have some different violations here and this is the one that I'm interested in fixing so this resource limits problem is a problem on a lot of clusters because a lot of people don't set the resource limits on their deployments and their pods so this is an easy fix that a lot of people can understand so if I go ahead and hit this fix button you can see all the different deployments and stateful sets that are

violating this so I can basically scroll down and go to anything but let's just pick this first one which is my Argo CD repo server and I'll go ahead and hit fix now if we have a look here armo actually tells us what the fix is going to be in our yaml manifest file so it presents us our current yaml manifest file and it says there's two changes that we should make the first change is on line 256 and the next one's on line 257. so if we click here it's going to bring us down here and here you can see it wants us to set up the resource limits so it actually gives us the configuration that we need to implement into our kubernetes manifest file so what you could do here is you could download this yaml file or you could just copy and paste these changes directly onto GitHub whatever way you want to implement it the important part is it told us what the problem was and it tells us what the solution is in order to get past this control you need to set your resource limits for your CPU as well as your memory now I did speak with the armo team and they did mention that they're going to be enhancing this specific feature in the future where it's actually going to give you an exact diff of your current configuration and compare it to the suggested configuration and I think maybe in the future they may even allow you to automatically push a PR with this change so a feature like that I think would be invaluable for a platform like this so I hope that makes sense let's go ahead and head back into the armal platform and now that we had a look at the compliance section let's look at the vulnerability section so if you go here you can see my clusters have quite a few different vulnerabilities and this is a big thing that happens in the security realm usually when you start running these tools your presented with a big long list of vulnerabilities and what happens to a lot of teams is they have no idea where to start you can see my clusters have 42 critical vulnerabilities and 352 High vulnerabilities now what I like about this platform is they actually tell you which ones actually matter so if we focus in on this section right here you can see although there are 42 critical vulnerabilities only 40 of them are actually fixable so these are known vulnerabilities out in the wild and actually only 40 of them have real fixes so even if we wanted to we couldn't fix all these different things the other thing it tells you is only one of them is actually relevant inside my cluster so this allows us to actually create a strategy around what we actually fix if there's 42 critical vulnerabilities but only one of them is actually relevant let's go ahead and focus in on what's actually relevant right so what we can do here is add a filter and click relevant and specify that we're only interested in vulnerabilities that are relevant to our cluster so you can see here this drastically reduces my list I have three different containers that are violating and uh these bottom two only have one violation in high whereas this specific container has seven critical vulnerabilities one of them is relevant as well as 50 high so I would probably want to focus my attention on fixing this specific resource so I could go in here and see all the different vulnerabilities and I could review them and if I determine that you know this isn't something that I'm going to worry about this is not something that I think is relevant I have the option to ignore it but if I want additional information you can click here and it goes into the actual cve explanation and gives you all the information that you would need to know about this vulnerability alright so let's go ahead and head back to the armo plan form and I'm going to go into the r back visualizer now so if I go in here I can select whichever cluster I want I'm going to go into my lab cluster as it has a lot more permissions so there's a lot more to visualize so this is actually a pretty interesting feature that allows you to visualize the actual

permissions in your kubernetes cluster so I'm just going to zoom in here and you can see that we have the cluster administrator it's so I can see all the different resources that have this cluster administrator cluster role attached to it so I can see that portaner has a service account and it's attached to Cluster admin I also have this Helm Traffic Service account and uh another attached to this cluster role so it's just a really good way to visualize who's getting permissions on your cluster now there isn't too much more I want to show on this particular feature I just want everyone to know that this is available for you to view your actual permissions and there's a bunch of different sorting you can do on it as well uh but what I want to get into now is the registry scanning and the repository scanning and how we can Implement cubescape and armo in our CI CD so the first thing I'm going to go over is registry scanning and this is something that's pretty simple to set up you just go in here and then you add your actual Registries that you want to monitor so this is a just a good way to hook up all your different Registries and have them scanned periodically so you can set up a cron schedule of how often your registry is scanned I'm going to leave this one here because setting it up is super simple and something everyone should be able to do what I want to focus on now is the repository scanning so if I go here you can see that I have two different repositories hooked up and the way that you hook this up is by running the cubescape scan command on the repository so when I first set this up I just grab this random GitHub repository I scanned it and you can see it returned me all the different resources this next one is this cubescape workflow repository that I created and this one was actually triggered by GitHub action so I'm going to go in here and you can see that it found at deployment. yaml file and it has seven controls that have passed and six of them failed one of them is a high severity for them are medium and one of them is low so this is the same thing that we looked at earlier when we ran the cubescape scan command on the repository but this one was triggered by a GitHub action so in the description below I actually have a link to my GitHub repository where I have this set up and it has the complete workflow of setting this up this is also available in the cubescape documentation if you want to follow that but this is a live example and I'll be going through how we can actually configure this to make it work so right now you can see my repository is in a passing state so the cubescape scans have passed let's go ahead and click this check mark and if we go into the details we can see the last run and you can see it was successful and if we actually go into this section of the CI workflow we can see exactly what happened here so if I scroll down there's that one high severity there's the four medium and the one low and my compliance score is 75. so a lot of you might be wondering why is it passing and that's because I set the threshold you can see the exact command that it ran right up here you can see the fail threshold is set to 70. so I'm at 75 so it's passing and I set the severity threshold to critical so there's no critical alarms so it passes now if I want to actually make a change to this configuration all I need to do is go into my code go into my workflows directory and modify this GitHub action workflow so this is straight from the cubescape documentation I just modified a few of the parameters here so I'm going to show you how we can make it so this check is a little bit more strict so I'm going to change it so if the severity threshold is anything that is High then it's going to trigger a failure I'm also going to say we need to pass at least with an 80 score so right now we know we do have a high severity

problem and our score is only 75 so when I make this change the next build should fail so let's go ahead and hit commit and I'll just save this and I'm going to give it one minute to run here and then we'll go ahead and have a look at the results now before I actually go there's one more thing I want to mention here that you may want to modify is this continue on error field so if you have this set to true it's always going to pass the workflow but if you have it set to false then you can make it so it actually fails the pipeline so that's just another parameter that I want to mention in this actual workflow in case some of you are working on this and you're running into issues so now that I've mentioned that let's go ahead and go into our code and there you have it we can see the last one failed so if we go here we can have a look at the details we can see that it failed and that's because we have the high severity and we're only getting 75 percent and now those parameters are set to 80 and the threshold is high so this is the same way you would sort of implement it in all your different CI CD pipelines you would just have them run a command similar to this with the GitHub action it makes it a lot simpler to actually just take the action that cubescape has built for us and then add the parameters that way now another thing that you have here is you can see that it gives you a link to the actual repository scanning and this is sent to the armo platform so if we just click we can see the exact failures here and now we have it available in the armal platform and it's just an easier way to have a look at it so if we want to see how we can actually fix this we can hit fix and let's see if we can filter on just high and you can see all these ones have passed this is the one high severity that is failing so CPU and memory resources should have a limit so similar to the one that I mentioned before and if we go in here we can see this is the suggested fix so basically what we want to do is set these resource limits in our actual yaml file so I'm actually just going to copy this entire object and let's go into GitHub and make the fix and see if it now passes the pipeline so if we go to deployments we should just be able to edit this and then just do a paste and we'll probably want to set some values here they just have some placeholder values this is something that you're supposed to come up with yourself so we can set these pretty low I'm just going to do uh 0. 5 of the CPU and then let's go 256 Mi so I'm going to go ahead and commit these changes right to the main branch and let's go ahead and see if the CI CD passes so I'll head on back here we can see that it's going through the workflow and we'll just go watch it in real time and I'll speed this up for the people watching at home and there we have it passed pretty quickly here so if we click this and then we have a look at the Run cubescape we can see that we no longer have that high severity we've taken care of it so now our pipeline is passing all right so I hope that makes sense if you want access to this GitHub action it's in the description below so you can check out my repository and have a look at my runs and hopefully that'll help you when you go ahead and set it up for yourself now there's at least two more things I want to show in the actual platform so let's head back here because these are really cool features so if we go down to the settings uh one thing we can do is we can set up our own framework and controls so if we go to Frameworks here you can see we have all the different Frameworks that come with cubescape and armo but at the very bottom here you can see that I created my own custom framework so you create your own custom framework just by going new framework and then you add some controls to it so if you have a look here I have one called production ready and this is just the controls that

I think would be applicable to my kubernetes cluster so I just put in the three most important controls that I think are important to me so you can see that I have the control executive container the resource limits and non-root containers so if I go down to my compliance tab I can actually filter on my clusters using this framework so I can choose it down here and then just focus in on it and you can see that my production cluster is not production ready because the resource limits has not been set so this is a good example of something you could do now if you want to create your own controls that's something I do recommend so if you go back to settings you can see that underneath the Frameworks button you have this controls and if you go here you can create your own control so let's go ahead and do that together here and you have a lot of different options here this looks a lot more complicated than it actually is basically you give it a control name so I'm going to say replicas must be two and I'm not sure if it's going to allow spaces but I think that should be okay but in this section we make a wish and basically you just explain in plain English what you think the control should be and it's going to go ahead and create that control so the disclaimer is it's provided by gpt3 basically it's using the GPT engine to automatically create this control for you so let's see how good this actually is I'm going to say replicas must always be set to two or more so basically for any of my deployments anything that has like a replica I want to make sure that someone needs to specify two or more we don't want replicas ever set to just one because we want some redundancy with our pots so I'm going to go ahead and hit generate control and see what it comes up with here and you can see that it created our control here so you can see we can download this control or you can just copy and paste this control so what I'm going to do here is I'm going to copy this down to a local file and I'm going to show you how you can use this control that we created uh using cubescape and we're going to run that against our yaml file all right so I've gone ahead and done that so I'm in this cubescape workflow directory again where I have my basic yaml file you can see this yaml file has three replicas so it should be compliant and then I just ran this command basically cubescape scan control I specified that control name for the control that I created and then I just copy and pasted that text into this control. json you can see it went through and it scanned that specific control and you can see the control name rough because must be two failed resources zero all resources one so my resource is compliant so this is just a quick example of how you can create your own controls so hopefully that was helpful alright so that's your tutorial on cubescape and armo I hope the video was helpful and I hope you use it to make those kubernetes clusters more secure if you have any questions about anything I discussed just leave a comment down below and I'll try to get back to you thanks so much for watching and I hope to see you all in the next video