Linux Commands in 30 Mins (Real-World Example)

Whether you are managing servers, deploying applications, or just troubleshooting issues on deployment environments, knowing how to work with Linux commands is absolutely essential for any engineer. It will literally 10x your productivity, whatever you're doing. So, in this video, I will show you a handful of fundamental Linux commands that will make your daily work much more efficient, but also make you feel really cool and confident about your Linux skills. But here's the thing, we won't just learn these commands in isolation with some boring use cases. We will build them up step by step, combining them into powerful command chains that solve real world problems. So we're going to go to zero to advanced within a short period of time, but in simple steps. So to set a context, let's imagine a realistic scenario where you are managing a web application that's been running for a while and it generates various log files every day and our task is to investigate error messages that users have been reporting. So we'll use this scenario to learn each command starting simply and gradually step by step building more complex operations using Linux commands. So let's get started. And in order to make your learning much easier, we actually created a handout for this video specifically where you will find all the commands that we are using here with explanations and examples that we're using so that after the video you can quickly reference that, refresh your knowledge and just keep it handy whenever you need. So check it in the video description, grab it from below and let's get started. So we are on a server where our web application is running and the very first thing we need when working with any system is to see what files we have available and that's where our first command ls or list comes in where we see that we have logs directory within our web application folder. So this shows list of directories and files in the current directory which is this one right here. However, we can also check list of files and folders in any other folder like this and it shows us that we have these three folders within locks directory. But let's say when I execute this command in the current folder, I want to see all the subfolders of those subfolders, any files inside them and so on. So I basically want to see the entire tree within the directory that I'm listing and for that we can use these commands with parameters. So we can modify the behavior of any Linux command by adjusting or tweaking it with parameters. So in this case I'm going to say show me a list of contents basically files and other folders recursively. And when I execute shows me that there's a locks folder and within locks folder right here you have these folders and within this folder you have these files two files and so on. So it gives me the entire tree within my current directory which is this one right here. So that's pretty useful if I want to see the entire chain. Now let's say in this folder I want to list all the subfolders that it contains. I don't care about the files. I just want to see the folder structure. For that I have a command called find which is very flexible command and again with parameters we can modify the behavior. So we can say in the current directory which is dot here. So in the folder where I am right now find all the elements which are of type directory and it will list all the subfolders. So just the folders, not the files. Okay, great. So now that we have the basics out of the way, let's go and do some useful things. So let's bring up our recursive display of all the files that we have in those subfolders and let's say I want to display all the files which belong to the application itself. Now, to make it easier for you to follow along this hands-on tutorial and to do exactly what I'm doing, I actually hosted all of these on a Git repository. So, you can simply download the folder with its subfolders with all these sample files that I have included here. So, you have exactly the same state as I have and the Git repository link is going to be in the video description. So, download that and then you can start following along. So basically in our web application folder

we have logs and for each month we have a dedicated folder. This is for January, February, March and then within each month we have the application logs and additionally we have some service logs right for readies for database and so on. So now let's say I only want to display the application logs I don't care about the other services. So I can do recursive search. So give me all of these and then filter the results which means I get the result of this command execution and whatever the result that I get I modify that result by executing another command. And in Linux if you want to chain multiple commands you use this pipe symbol. So basically the output of the first command becomes the input of the next command. So it kind of goes through a pipeline or a pipe so to say. And here I can execute a command that will use the output of the first command as input. So I'm going to say take this result and filter it using command called grap. And I can define what the filter is. I can say give me only the app. ext files. And let's say if I had 12 folders for every single month and hundreds of services and lots of different files in those directories, this will give me a clean overview of only those files that I'm interested in. And an alternative to this command right here is actually find that we used before which is a more flexible command. And I can say find in the current folder where I'm right now that's the dot here. All the files type f. So I don't care about the folders. I only care about the files now that have a name called app. ext. And now instead of just getting the name of the file, this actually gives me the entire path. So it shows me where exactly each of these files is located. So this gives me a more informative output of where all these files are. And right here I can say give me all the files that have txt extension. I don't care if they're application files or service log files. Give me everything that ends with txt. And I can do that with a star symbol. So this covers anything. This is a regular expression that tells Linux find all the files recursively in the current directory and subdirectories where the name is anything. ext. And if I execute this, it will give me all the files in all the subdirectories that end with. txt. And this is extremely useful when you don't know exactly what the files are called and you have hundreds of files with different extensions and you just want to get files of the same extension regardless of what they are called because they may be called differently and you just want all of them or you may not know all the names. Now, before we continue, I want to give a huge shout out to F5, the company behind EngineX, and this is not your usual sponsorship shoutout. F5 wants to clear the confusion around Ingress EngineX controller, which is retiring. Given how widely Kubernetes is used, many teams running Ingress EngineX are unsure what this actually means for them and what their options are going forward. So, let me clear that up real quick. The decision to retire the community-maintained controller came after years of the project being maintained by just one or two developers which basically caused serious security issues last year. So what's the alternative? Ed5 actually maintains an open-source Apache 2. 0 licensed Ngx ingress controller and yes it will stay that way. Plus all development happens publicly on GitHub. But what's different and important is that there's a dedicated full-time team ensuring regular security updates and feature development. Now, if you're worried about migration, don't be because it uses the same EngineX engine you already know. Most Ingress EngineX annotations have direct equivalents and there's a migration guide with clear mappings, so minimal learning curve. In addition to that, it's production tested at scale, powering approximately 40% of Kubernetes ingress deployments with over 10 million downloads. So, be sure to check out the detailed blog post at blog. engineext. org for more in-depth information on that. I

highly suggest you get started on that. And you can simply get started by pulling the image from dockerhub enginex/inex ingress and try it out yourself. All the relevant links and information is going to be in the video description. All right, now back to our Linux commands. Great. So now we can find all our text files across all directories. But what we actually want to do is look inside these files to investigate the errors and issues that our applications or the services have. Right? Just listing those files doesn't give us much. We located them, but now we have to actually investigate what the issues are inside of them. So one way to display contents of a file is using cat command which stands for concatenate a fancy word for basically displaying a chain of characters or strings. And since I have the location of the file I can copy this entire path so to say. So the first dot always stands for the current directory. So we're starting from the current directory and the slash is for subfolder. So in the current directory web we have a logs folder and inside locks we have this folder and inside this folder we have this file. So this is always current directory and if I execute you can see that it prints out the contents the full contents of this file. Now the problem is I don't want to execute this command for every single file. Let's say we have 10 files here. I don't want to run this cat command and then copy each location. I actually want to have one command that gives me contents of all the files in this list. Or let's say I want to display contents of all the files which are called app. ext. So right now we don't care about the services yet. We want to investigate the application blocks. So how can I display contents of all app. txt files in all the subfolders with one Linux command. Let's clean this up with command L. That's a shortcut for cleaning up. Or on Windows, Ctrl L. So again, we're going to find all the files recursively. We type F that are called EP. ext or txt. So again, this will give us the list of files. And now we want to display all of those. So we're going to remember chaining of the commands or piping them. So we have the results of this first command that we can give as input to the next command by using the pipe and the command that displays the file contents as we learned is however if I execute it like this was actually try you see that it doesn't give us anything right this one executes and this doesn't display the contents because we have multiple entries here so what we need to do is we want to take the output from the previous command. So this whole list here and turn it into individual arguments for this command to execute one by one. And in Linux there is a command for that called X arcs. So this basically turns the result of the previous command into arguments and passes them to the next command whatever is defined here as arguments or as parameters input. So if I execute this we can scroll a little bit. So these are combined logs from all three ep. txt files. Awesome. So we're displaying contents of multiple files with one command and we can do the same for all five files actually including the radius and database and we see the entire output here. Now this is great but a little bit unpractical because we have lots of lines of logs and it's a little bit difficult to read through them and we do have errors here but we also have info and warn and some other stuff a lot of noise basically. So, how can we remove all these extra information that we don't care about and only display the error logs or error lines basically? So, we can zoom in to the issues and remove all the other distraction. And that's where we're going to extend our chain of commands even more using pipe again. So we take this as a result whatever this gives us and filter this entire output again using grap command and we say only display lines that include this word. So any line that has error inside display only those filter out everything else. And let's execute. And

as you see that's what we're left with. So in all five files, it basically gave us only the relevant log lines. I'm going to zoom out a little bit so you can see the logs a little bit more cleanly so to say on one lines. You don't have to read any of this. It's just random generated data. So don't worry about what the error messages are specifically. But now we see that we did filter it but there's still lots of locks here and especially there are some duplicate ones right we have Reddit connection refused. So when a service has an issue or application cannot connect to a database or a service it basically just goes into a loop and prints out the same error over and over again right with different timestamps. So let's say we want to see all those duplicates together. Like these ones here are multiple times. Database connection failed multiple times. So we want to kind of put them all together so the view is even cleaner. How can we do that? Another pipe. So we're going to say take all of these as an input and just sort it alphabetically. But here's an interesting thing. Let's execute this and see what happens. So I'm going to sort. So this command should put all the duplicates together. But if we look here, we have radius connection refused. Another one, another one. So they're not really listed together, right? What it did was it actually sorted per time stamp. So now we have all the errors with times ascending compared to before, which may be useful if you want to follow a specific timeline and see where all the errors happened. But we specifically want to sort the errors to group together the duplicates. Right? So how can we do that? How can we tell Linux do not sort the date or these first two columns so to say, but instead sort this part here. Well, sort command has a parameter. Remember Linux commands are flexible because you can tweak the behavior of them using various different parameters. And sort has a parameter called K which basically allows you to choose the field or column. So you see these spaces in between. So this is one field, this is a second field, third field, fourth field and so on. So we can say sort starting from this one. So ignore whatever is here. You don't need to sort these columns or fields here. You need to sort this one, which is the fourth field, right? 1 2 3 4. So, we're going to say K4 and execute. And now, if you go back, you see that it's alphabetically sorted. So, it starts with A B C and all the duplicate ones are grouped together. So, instead of time stamp, it sorted this field. So now let's say we only want to see the unique errors so that we can reduce the number and we can concentrate on what is actually happening in the systems. What type of errors we have we don't care that it's happening four times or five times. And for that there is another command and you guessed it right. We pipe it again. So whatever the result of this chain of command execution is, so all of this we pipe it to another command called unique. And we basically say only give me the unique lines from the logs. And this is also interesting because if I execute this, you see that it does not filter out or give me only one of those because it actually looks at the entire line whether this entire line is unique or not compared to the other ones. And you see that timestamps are different of course between those logs because they happen in different times. And again, you guessed it right again. We have a parameter that lets us modify that behavior saying we don't care about uniqueness of these fields. We this part only or these two fields basically. And the parameter for that is f. So we're saying start from field number to check the uniqueness of the line. So we can say check from here or from here. Let's say this field here which is number three. So we can say check the uniqueness of a line starting from this field. So these values will not be compared. And now if I execute you will see that all the duplicates are gone. Right? And we have only unique errors. And you see that number of errors actually became way

less. And we're checking all five files by the way. And there you go. Okay. So we have this super long chain of commands. But every time we want to check this specific output of these files, we don't want to remember to execute this chain of commands. We want to save this somewhere so we can reference it as a file. Right? So all of these we want persisted in a file that we can open in an editor maybe in a more user-friendly view instead of having it here in the terminal. So we can tell Linux take the result of this entire command execution. So this final result here and save it into a file. And it's actually very easy. But this time instead of pipe we actually have this greater than sign. It's kind of like a different pipe which is a symbol that basically redirects output of the last command into a file. And we can call this file whatever we want. Let's say errors. lo. And if this file does not exist in this current directory, it will be created with the contents of the last command execution. If it already exists, it will be overwritten. So we don't have this errors. log file in our web app folder. So if I execute this, it will be created right here. And remember how we display the contents of a file using cat command and that's where we have this output here. Now as I said if we use greater than sign and the file exists already the contents of the file will be overwritten. So you won't keep anything that you had before. But sometimes we don't want to overwrite things. We want to just append. Right? So let's say we analyze logs of March or April and we want to add them on top of whatever is here already. And for that we use double greater than signs which instead of writing from scratch it just appends to whatever existed before. And now if I execute this again and display you will see that we have it twice. Right? So we just so it appended the result again. And if we want to override that and create it from scratch. There you go. Great. Now we have our errors log file that includes all the relevant errors. And let's say we analyzed the issues of our applications of our services. We fixed the issue and now it's time to move on so to say. But let's say, let's actually show them again. But let's say we have an automated script running on a server that deletes some of the older logs. So let's say after a few months, for example, so let's say in April or May, these older log files or folders will be deleted in order to free up some space on the web application server for the new logs, right? Because obviously we don't want to run out of storage but those logs actually have still valuable information. So we don't want to lose the data. That means we want to create backup of all the log files of our applications of the services let's say on another server which is dedicated just for storing backup data. So to mimic that, let's actually create a backup folder in my user's home directory using make deer command or make directory command. And I can say in this folder create a directory called backup. And we can check that it was created. There is nothing inside yet. We can put all the files. So we can copy or back up all the files from here into this location. So ls or list command gives us empty. If the folder did not exist then it would just say no file or directory. So with this command you can also validate that the folder exists or it's there. Okay. So how do we copy multiple files into specific location? We could copy individual files using cp copy command like this. So we can say from current directory again logs this subfolder take this app. ext file and copy it into backup directory and it's going to be copied there. But let's say I have again hundreds of files here

and I don't want to execute this hund times copy pasting the file name the backup location name and so on. So how can I do it in one command? Well, we could use our familiar X arcs command which loops through the list and provides them as individual arguments to the next command. But we have a more elegant alternative to that actually which is a parameter of the find command called exec. So this find command is searching for all the files in the current directory that are called something. txt and this will return the list of all the txt files. Right? So now we can add a parameter. So we're not piping to into another Linux command. We're actually using the same find command, just tweaking its behavior, modifying it with an additional parameter called exec. And what exec does is basically tells Linux to execute a specific command for each file that it finds. Again, this is specifically parameter for find command. So for every file or output that it finds here execute the following command and the following command is copy right. So every file that was found here with this command loop through one by one and for each one of them execute copy command. So copy each file and each file can be expressed by these curly braces. So this is like a placeholder so to say that references each file that this command found. So on the next iteration this file will be substituted here. So this is like a placeholder for each item in the list. And if you know loops from program language that's basically what this does. So it copies each file or each item in this iteration into the backup folder. An exec parameter requires a symbol that marks the end of command execution. So we need to tell exec this is the end of the command that needs to execute for each item in the iteration. And we say that using a semicolon. So this basically tells exec this is the command section. It's like a again from programming it's like a block of command and this is the end. But we need Linux to interpret this as a special symbol. And for that we need to escape this. That's a concept that basically tells Linux do not take this literally and directly. Do not treat this as a semicolon character. Instead, special character with a special meaning for exec which the meaning is that it denotes an end of command execution. And that's where the backslash comes in which tells Linux this is a special character. So that's the concept of escaping the literal meaning of a character. And now if we execute this, let's see what happens. And we have a very interesting case here which is you see that only one. ext file got copied. Why? Because we're copying all these files in one directory. And a folder cannot contain two or more files that have identical names. It's going to create a conflict. Like if I try to create another backup folder in my home directory, it's not going to work because it's going to say backup folder already exists. Same way if I try to create app. ext inside backup, it's going to say app. ext already exists. And for that there is a command called rs sync which is actually a command very widely used during backup operations or for backup scripts and we can pass it a parameter for recursive as you know already and this will preserve the parent directory so to say of those files when copying them to the backup folder. So let's execute. And now if I ls and let's again display recursively or use find as you see we can do logs directly. So it created this logs directory inside backup folder and preserved exactly the structure that we had before using year and the month and the file name. So, this is actually an alternative and actually more powerful command for backing up files, moving them around, copying files and folders on your Linux systems. And we can actually go into

our backup folder and we can remove all the text files using rm or remove command. And instead of listing them one by one the app uh database and radius we can again provide this regular expression and now as you see only the logs folder and its subfolders remain. Awesome. So we have analyzed our logs. We have printed out all the error messages in one place to quickly identify any issues. We fixed the issues in the applications and then we took backups of all the log files in case they get removed or deleted from the web application server as part of the cleanup. And as you saw, we did that by taking simple Linux commands but combining them in very powerful and flexible ways. And that is actually a big concept in Linux which is the power is not in the individual commands that you know but how to combine them for different operations where each command does one simple thing but together you can combine them to solve more complex problems. So small focused tools that work together and that's why Linux is so powerful for engineers and you can do a lot of automation with Linux commands. So go ahead and practice this. Combine even more commands. Think about the use cases first. What is it that you're trying to solve? And then find the commands or chain of commands which help you execute exactly that task or solve that use case. I hope this video was very helpful for you and your confidence level in using Linux actually increased after doing this video. If it did help, then make sure to share it with one friend or colleague who you think needs some Linux skills so they can practice as well. And with that, as always, thank you for watching and I'll see you in the next