ChatOps: How we automated the mundane in 2020 with n8n 💭

hey everyone welcome to my talk on chat ops how we automated the mundane in 2020 today i will take you through my journey which started in mid-april last year and let me deep into this world of low code tools and automation this talk is divided into three parts the what why and how of the story i'll also give you a brief introduction to na10 the tool that we use for automating our processes and share with you the workflows that we build before we start a brief introduction my name is tanepand and i work as head of developer relations at na10 i'm really passionate about automation local tools and devops before na10 i have worked on a lot of devops and data related topics around automating smart factories other than that i love writing and have published books on firefox os virtual assistants and web vr i joined naden last year around mid april and i was fascinated by the diverse use cases for which the community was using this tool i talked to our community to get a better understanding of what they were using automation for and why they were doing it so the answer to what was diverse people were using automation for devops monitoring marketing social media iot backend of applications and even on ships this really blew me away now the answer to why was a bit more coherent and the common thread was making things efficient quick and easy and this made sense this wasn't very different from what i had seen when working with automation in smart factories one day i was chatting with the founder from our community he was working on his startup doing freelancing and he worked full time as a management consultant for him automating things was primarily about maintaining mental health he pointed out various mundane tasks that are repetitive and performing them day after day was a drag for him sure efficiency was a factor here too but taking care of his mental health was most important to him and this was an interesting revelation and something that i could relate to as well we all do things daily that can be automated and can free up so much mental ram inspired by this i set about to recognize these mundane things in our team's day-to-day life and there wasn't a shortage of that part of being an early stage startup means that every day we'll have new processes new projects and things that can likely be automated now the first step of doing that was recognizing the processes that were already working well there is no value to be gained by automating a broken process after all so three prominent use cases immediately stood out and there was a common thread among these were things that we initially did not foresee as time consuming over time as we grew and started accelerating our efforts these became mundane tasks and we had other priorities as well so the first was pushing new ntn

versions to the cloud so this involved marking the new version as peta marking the previous version as table and removing the version before the previous version from the control panel this involved working with the postgresql database the second was automating the generation of release graphics so we have a new version released pretty much every week we have a graphic with logos of some of the new integrations which is published on social media this involved things like compositing images and converting 2d icons to isometric images and finally we wanted a weekly update on the documentation coverage this meant checking which integrations have not been documented which need updating and so on over the months we built more automations like these and that gave rise to the question how do we manage all these from a central system and get information where it's most accessible with the least amount of friction we also wanted to have fun while working with all of this so that led to us embracing chat ops so we use mattermost which is an open source alternative to slack we built a couple of slash commands and buttons and voila all the information that we needed flowed into matter most and we could trigger things to happen from within matamost as well so my tool of choice for doing all of this was na10 for those of you who don't already know about na10 it is an extendable workflow automation tool let's take a look at it na10 has a fear code distribution model which means that it will always have visible source code be available to self-host and allow you to add your custom functions logic and apps its node-based approach makes it very versatile enabling you to connect anything to everything now there are a couple of different ways to run an a10 you can run it using npx npm docker or sign up for an account on anything cloud this documentation on each of these ways in the quick start section of the docs so let's do a deep dive into na10 so first of all we have the editor ui so this is a blank canvas where you can create your workflows then we have nodes are the basic building blocks of a workflow in na10 there are two different types of nodes trigger and regular nodes so the trigger nodes can start a workflow and supply initial data a workflow can contain multiple trigger nodes but with each execution only one of them will execute an example of that could be the type form trigger node which will start a workflow when let's say somebody submits a form then we have the regular nodes which do the actual work in a workflow they can add remove and edit the data in the flow as well as request and send data to external apis they can do everything possible with nodejs in general an example of this would be the motek node which can deal with resources like company and contacts and handle crude operations let's create a new workflow before delving into the use cases we talked about earlier when you create a new workflow in na10 it will contain the start node by default the node cannot be deleted so in case you're not using it you can move it to the side so let's say we want to monitor a website by checking every hour if it's up and running in case the website is down we want alerts in our chat tool right so let's get started so first of all we will add the cron

node which is a trigger node since it would be starting the workflow so we add

a cron node to the editor ui gonna place it right here and since we want it to run every hour i'm gonna change the type of mode to every hour clicking on execute node runs the node but this node doesn't do anything by itself so we don't have any results here now we want to use the http request node

to make a request to a website so for that i'm going to use the http request node so as an example of a website that we need to check let's use the docs for n10 now the response that would be returned would be a string because it would return the html for the website so if i click on execute node you'll see that it returns the html for this website as the data now i also want other kinds of information um so i click on full response so that i also get the status code error message and so on and when i do this you'll notice i have some more things here like the headers as well as the status code which right now if we scroll down here is 200. now in case the website doesn't exist that will return a 404 but that will cause an error in the node so because we do not want this node to error out in case there's a response other than 200 we are going to click on ignore response so once we are done with that we want to see if the response code that we got is 200 or not so for doing that we need to implement

conditional logic and we'll use the if node here so we're going to add the condition we we're going to be comparing numbers here so we're going to select number and we want to see if it is equal to 200. now the value that is going to come in from the previous node would vary depending on the url so this would be a dynamic value and for that we'll use expressions so we click on add expression we'll delete 0 click on current node input data json and here we see the data that was outputted by the previous node and is entering this node and we select status code here so this is an expression which for this execution and for the url that we provided gives 200 and we execute this and see in the true output we get all the data that the previous node output in and for false we have nothing okay so when the status code is 200 and the response is true we want nothing to happen so i'm going to add the no operation do nothing node here now adding this node doesn't really make a difference to the workflow it's here primarily for aesthetic or cosmetic purposes and it helps uh visually understand okay if it's true nothing happens but if you skip this node uh it wouldn't make a difference to the functioning of the workflow now to the false output we're going to add a mattermost node so that it sends a message to a channel every time a response code other than 200 happens so i'm going to connect this now all the nodes uh which are talking to an external api that require authentication have option to enter credentials so for instance the mattermost api asks for things like access token uh the base url and in case you are unsure where to find these details uh you can go on the top here and click on open credential docs and these will show up the instructions for finding this information okay so i've already entered the access token and i'm gonna select this now we need the channel id of where to post this information so i'm gonna open up matamost and grab the channel id from there i'm gonna copy this paste it here now the error message could be error occurred okay now let's execute this workflow you'll see that since the response code was 200 nothing really happened now what if the url is incorrect or the page is not found so let's change this to quick start one and then execute the workflow so if we take a look now on mattermost we will see that we got the message error record because it probably got a 404 as we see here now to make it a bit more specific um let's add expressions so that we can specify which error exactly happened so we can say here response code and then from the previous node we can actually put the response code as dynamic data and then add status message as well and for status message we're gonna add what exactly it found so let's execute this node again and now if we go to mattermost vc it says response code 404 status message not found so right now we were doing manual executions by clicking on the execute workflow button but we want this workflow to run hourly and that would be making it run in production so for that we'll just click on the active uh toggle on the top right for doing that we first need to save the workflow so gonna just name it check uh website status save this workflow and then activate it so now this is gonna check this website every hour and depending on if there's a error if the response code is 200 or not it will share the error in the matamos channel now let's do something else what if we

want to manually check if a website is up and running from matamos so let's create a new workflow by copying this one so you can copy workflows download them and share them because all the workflows are essentially in json format so i'm gonna do command a command c and then your workflow then i'm going to do a command v here and it's going to paste the workflow here perfect so first of all we'll have to replace the cron node with a webhook node for doing that so we click on webhook which is a trigger node so we find it here so the webhook node would be the trigger node for this second workflow that we have so let's save the workflow and we're gonna call it slash check slash command so i'm gonna save this and now the webhook has two different webhook node has two different kind of urls so the production url is used when the workflow is set to active but what happens is when data is being sent to the production webhook url it doesn't show up in the editor ui so it's not great for testing and debugging so we're going to use the test webhook url for testing and debugging here so i'm going to copy this now we are going to create a slash command in matamost so we can do that by clicking here going to integrations slash commands and we create a new slash command we are going to call it check and the trigger command word also we check the request url would be the webhook url test webhook url that we received and let's ask it to autocomplete um call it check of website and we save it perfect now we can go back to matamost now what we can do is if we click on first of all we'll have to change the http method here to post because that's what the slash command is configured for if we click on execute node and we click on slash check hello as i press enter you notice that this workflow is executed and we got some information here perfect so now that we have our information um how the command would work is we would actually say check and https and some website so this slash command would actually contain the url of the website as well so we have to edit our http request node so that it gains this url from the web hook node so we're going to edit this here and add an expression and this is the text that we are getting we wrote hello but this would actually be a url and everything else remains the same i'm gonna save my workflow i'm gonna click on the execute workflow button again and gonna enter slash check and we've got no response so that means that the website was a 200. okay what if we enter a wrong url so let's take this same url we took the last time and execute this workflow so we do a slash check and let's say quick start one. html and we got another message response code 404 status message not found so this is how you can create slash commands uh for whatever chat tools that you're using maybe slack rocket chat matamos and now let's take a peek at the three automations that we talked about earlier and how they look like in n10 the first use case that we talked about was pushing new na10 versions to 2810 cloud so this is handled by two workflows so this is what the first workflow looks like it's initiated by a webhook node which essentially looks for new releases on github so whenever a new na10 version is released on github it will trigger this workflow then we set some information like the name of the version is it stable is it available initially both of them are set to false and then we do three things first of all we post a message to the announcements channel like hey there's a new version of nh10 available do you want to release it to staging production then what we do is these two nodes do pretty much the identical function but they are talking to different databases one is talking to the staging database and production database if we open up this node we'll see that we are inserting these uh new unit and versions to a table called na10 version and we are putting in information like name is available and is stable now let's take a look at how this announcement actually looks like on matamos so you see we say ntn core release is the version number and we have then two buttons release to staging and release to production let's take a closer look at the matamos node here so with these two buttons we also have an integration attached to it so clicking on these buttons would essentially trigger another workflow and this is the url of the web hook of that workflow along with this button we can set certain information as well like here we are setting does this button mean a version should be released to staging or production and what's the version of this nhn release and we have similar settings for release to production button so we can save some context and we can tell it which webhook to call all right so once a user clicks on one of these buttons it starts this workflow so this is a relatively longer workflow let's take a look at what it does so this workflow is started by webhook which is activated or triggered once a button is clicked and now since the webhook gives in certain information from matamos as well like what's the username of the person who called this workflow along with the context like uh do we want to release it to production or staging and what's the version number after that we have an if note so we want to check actually if the person who clicked on the button has the permission to release a new na10 version to cloud so here we check and we see if one of these three people if the username matches then this workflow continues otherwise it says like sorry you don't have the permissions to do this at the moment you know if you feel like this is an error reach out to these people so in case the user is permitted and start acknowledgement is posted on the channel so it says release of this version to staging or production is triggered by this person so this actually looks like something like this release of 104. 2 to staging triggered by an then this if note checks if the information that came into the web hook node is it production or is it staging so based on what the value is and based on which button was clicked either this or this it takes one of these two parts and the paths are pretty much identical except that they are talking to different database one to the production database and one to the staging database so here we do a couple of things such as like we get the version we check if version exists in staging and this might happen like there is no version because uh the previous workflow failed so essentially uh we added a couple of fail safes here so in case you know a minor version is really is like 0. 14 104. 2 or 104. 3 or is it a major version and just making sure like nothing fails because this is making a change on n810 cloud so other than that what we do here is make the version previous version stable make the version before the previous version unavailable and like we have added some custom javascript code to it as well so you can if you have to do some things which are not possible with nodes in na10 you can use the function node where you can add javascript code and uh once this is done it puts a success message in the channel so it says released version this to staging by on or released version to production by on so this is how we use na10 to build n8n cloud so the second workflow that we talked about earlier was automating

the generation of release graphics so this is the workflow that handles the automation of those graphics so before trying to understand this let's actually execute this and then go through step by step to see what these different nodes do so i'm going to click on the execute workflow button i'm going to open up matamos because this is a slash command as well slash graphic and here we give it four pieces of information and all of these are separated by a comma so first of all we have the new version of naden and then we have three urls of images which are in a square format so these are essentially logos of the icons that have been added to the new na10 version so i press on enter and you see it executes this workflow and it's generating the image now so let's start with what triggered this

workflow so here we have the webhook node which captures the information that we gave by the slash command then this function node what we do is we take the string that we gave it which was separated by commas split it into arrays and into a format that n10 can handle well so you see it changed the three urls that we gave into three separate records here essentially here it downloads images from these urls so here you can see we have three images and here's the three logos that we provided it in the url format then the next node resizes it so it makes sure everything is 400 pixels into 400 pixels now this is a multi-step node what it does is nest the data or the image into a format that will composite nicely on our final graphic then we do work on changing the 2d icons into a isometric graphic so this actually we resize it first then we do shearing and then we do a rotating action so the image ends up looking something like this so this is not 2d and straight anymore it's in an isometric uh format and from here we actually get the background of the image which is stored on next cloud and like different assets and finally what we do is we composite these on top of each other and we end up with a graphic that looks like this so here we have the text that was provided in the slash command and the three urls that we gave this workflow took them uh resized them and converted 2d icons into isometric graphics and produced this so this is how we generate our release graphics every week the last automation that we talked about was how we use na10 to monitor coverage of our docs because we want weekly updates on which doc which nodes have not been documented yet and also which ones need to be updated so we use this workflow for handling that so this workflow is triggered by another slash command slash docs what it does is it pulls in data from a graphql endpoint that is exposed by our website so we have a page entertain dot io slash integrations which lists all the integrations that a current version of n8n has so it also has a graphql endpoint from which we can get a list of okay what are all the different nodes that are in the latest version so we use that data to make an http request and the url of the docs is essentially dot entertain dot io slash nodes and then the name of the nodes and this name we are getting from the graphql endpoint and now we are doing here very similar to what we did in the workflow that we created earlier on is we are sending a request and seeing if it's a 404 we know that a certain node has not been documented yet and that enables us to know every week which node is not documented yet and what's the coverage looking like so let me show you how this slash command actually looks like so here we have slash docs so right now we have around 250 nodes so it's going to send a http request to all of them and see if there's 200 that node has been documented if it's 404 it hasn't been so here you can see that we have the number of undocumented nodes 10 current coverage is 96 percent so 252 out of 262 have been documented so this helps us keep track of how's the coverage like we also have alerts if it drips below 95 percent uh then we get alert for our team it's like okay the coverage is going low uh let's get that moving so these were some of the things that we automated using n8n and that has helped us a lot so as the months passed by and we integrated more and more of this automation into matamos over time automation became second nature to us finally it transformed into a maricondo-like philosophy does it bring joy no do we have to do it very often yes then that's a task that we should automate well that's all for today thank you so much for joining me and if you have any questions feel free to send them along in the chat you can always reach out to me at the nhn. io we also have a very active community on discourse feel free to join us there and i'm really looking forward to seeing what you all built thank you so much bye