Building an Enterprise-Grade n8n Dashboard with Real-Time Monitoring [#14 Aemal Sayer]

What are we building today, brother? — We're building a dashboard for anything. Something that enterprises love to have to keep an eye on every single execution of their workflows. Not only seeing what are failing, but also why are they failing to get notifications and even tweak it in a way that you know how much each workflow costs you, especially with tokens and everything. So, it's going to be a lot of fun. I personally trained over 8,000 people in it and is going through this code. It's taking all of that data and it's calculating some total duration of this N10 workflow and then it produces a bunch of data. If you're not using it, you're leaving like I don't know 10x 30x productivity on the table. How long it took me to build something like I call it vibe kidding. I built it for my son. He is 7 years old. — What is the value being generated in terms of the overall operational capacity of this init workflow? How many executions are being run? If people want to get access to this, get up and running with this, use this system, and what do they do? Amal, what are we building today, brother? — We're building a dashboard for anything. Something that enterprises love to have are failing, but also why are they failing and get notifications and even tweak it in a way that you know how much each workflow costs you, especially with tokens and everything. — Awesome. I'd love for you to share your screen, walk us through what you have coming up, and just to let anybody know, this will be made available showing you exactly how you can download the whole GitHub repo, make it available to you, and show you how you can set it up for yourself. So, by the time you walk away, you can have your own enterprisegrade activity monitor system within. — Absolutely. So, the whole concept is very simple. Something that we at a love to have for our clients, usually enterprise clients, having absolute monitoring in every single thing that you're moving like in your workflows. So the idea would be very simple. We will have a very simple NA10 instance running. I will do it just locally to make the point but it also works on the Nan enterprise license and then we'll have a superbase instance running in between that will basically take all the data from this Nan instance and store it inside the superbase instance and then we will have a rilet front end built from that data. So very simple like

three basic steps. So in order for us to stay ahead of the time and not face any hallucinations of LLM's code generators, I would like to like literally jump on the code base and hopefully anti-gravity will be nice to us. Just started it two days ago coming from Cloud Code, Gemini CLI and even actually built a little thing myself called Vibe box. put together multiple of these ones, but let's start with anti-gravity. So, I'm going to open source everything at the end of the podcast and you'll get a repository in the link of this video so you can get start running on that. So, I'm going to create a directory call dashboard. So n dashboard cursor let's go with gravity this time. So I span off gravity anti-gravity. So what we will have is eventually a bunch of files that will help us like spin off our docker container and so on. So the very first one we need is a docker compose file. That's not a big deal. It's something that probably you all know. It's a very simple instance of an anything I'm spinning off in here. What we're going to touch upon is something called hooks. This is not web hooks. It's an anything hook. Anything under the hood has this um hooks or backend hooks. And these external hooks or backend hooks have certain events. The moment that event is triggered, the anything code under the hood will call one of your JavaScript files and one of these the list of these events that you can see. The one that we are interested in is post execution. This one will give you a way to uh get literally all the data, the workflow data. And that's exactly what I'm looking for because when I have an N10 [clears throat] workflow and I execute it, it will produce some data and if you capture that JSON, then you can build a lot of workflows on that. — Yeah, I would I describe this as a telemetry data allows you to see the actual operations of the system, not the actual the difference between a web hook and a hook, right? The web hook is bringing data into inadin that allows you to transform, manipulate it and then send it through on the outside whatever you might be doing with it say to a Google spreadsheet or a superbase or whatever you might be doing with that data. These hooks give you telemetry data that let you know how the ina in actual system is operating in terms of activities and any lost data or workflow executions. And so this is very useful especially if you are a technical user and you want to know how are people using your systems whether you're the CTO or somebody else who is onboarding other users to use the system or if you might be a business owner operator that wants to peek in at a very high level to see the dashboards of how is my in environment handling what is how many executions are being run what is the value being generated in terms of the overall operational capacity of this inadin workflow well in environ enironment. — Absolutely very well said. So you can have a graphana dashboard integrated with your an data but this is a way to take that data and then do something with it. And additionally we're going to use riplet to build that dashboard by ourselves. So I'm going to continue bringing some of these files from the other system I have. So we need an environment variable with all the environments set here. We're going to bring a simple git ignore file to make sure that our git ignore is our git repository is clean and nicely done. And additionally, the most important part is that we're going to have a hooks folder in here. And then there will be a file inside this that's called execution hooks. js. It's a JavaScript file. I'll explain every single bit of this code in just a second. So, it's a simple JavaScript that exports. It's a basically a module uh export and then it has this object where every single element of this object is treated as a an event and it's called. You can see that this is a self uh a self-invoking. It's a function basically an anonymous function that's executed the moment the NAN instances ready in the same way the moment the workflows do one of these activities activated created updated pre-executed post-execute and so on and so forth. In case of post execute, you can see here this is an array. It has bunch of it can have a bunch of functions. But the function that we are looking for in here is the function in post execute that has this parameter that's interesting for us the workflow data. So the moment NAT completes executing an NATO workflow, it basically passes all of that data that the NATO workflow produced inside these parameters, these three parameters and then we can take and do something with it. As you can see here, what we are doing with this data, this code that I have generated previously with AI obviously is doing going through this code. It's taking all of that data and it's calculating some total duration of this N10 workflow and then it produces a bunch of data. This telemetry del example that you give. So we can literally push this to telemetry. We can push it to anywhere but in this particular case we're pushing it to a simple basin in just a second. If you zoom in and see it's just the workflow ID, the name and so on and so forth. And

not only we log it on the console just for some visual purposes and also you can go and check the logs later but also we are calling this function the function that we had defined in here. What this function does it basically passes this all information inside a table called inn execution logs and then it basically saves the data. So the moment the execution is done from this instance, it is automatically saved inside that superbase table. And how does that table look like? I will [clears throat] leave the schema here. Super simple. A few columns, a few indexes, and that's it. So, I'm going to just copy this, go to my superb basease, and open an instance. I already have this instance. And I previously created this table. So, I'm going to just kick it off to give this example from scratch. And I'm going to just execute this entire schema. So, what this table has is we can also enable the R level security. So what this table has is all the columns that we are looking for have produced on the front end or in not in the front end in that middleware in that JSON that JS file that we have created this execution hooks JS file and now once we have spun off our N instance locally you'll see that the data is stored here so let's go ahead and check that out so here my docker demon When it's running, all I'm looking forward to do is spin off my Docker container here. It should spin off my Ant instance. — Nice. And as we're doing that too, one thing I want to say is it would be good to have the SQL be made available. It could be inside the GitHub or whatnot. And maybe we can do is take the transcript from this podcast and turn it into some instructions that we can then populate back into the GitHub file or GitHub project so that you're able to go ahead and go through it and follow it step by step. and then download the resources needed. — That's an amazing idea. So basically there will be a readme file here that will come from the transcript. That's amazing. I like the sound of that. — So actually let me quickly create the Nin repository so that I can keep pushing as we go on so that we don't lose anything in here. Sign it in dashboard analytics. I'm going to create a public repository out of that. And this is not yet. There we go. So we have now the entire codebase that I showed here available publicly inside this dashboard analytics and as you mentioned the readme will follow from the transcript. — Fantastic. — All right. So we have the initan instance running locally with us here. I'm going to quickly go and create a simple password here. It's my local so I don't need to really be worried about the security of that. And I'm going to throw this away anyways. So once I'm done now we have started our session. Basically the session is already being traced or tracked using this particular hook and this hook is sending data to that in instance this basic instance if all the wirings are done properly and I go and create a simple A10 workflow and let's say if I do an HTTP request to a JSON placeholder pick the users API which is basically just a website that provides these mock endpoints or APIs and I will use the users API. I'll copy this URL and fetch this data. The moment I fetch this data, you see what happens. I have executed an anything workflow. So everything was successful and it produced a lot of data and also if you see the execution tab some execution context was created and what happened behind the scenes was that the entire data went through that workflow and it ended up into this database. The execution ID, the workflow ID, the name of the workflow, what was the status and so on. But most importantly, what was the execution data? This execution data — is a JSON. And if I open this here, you can see that this is exactly that JSON that is basically generated by that anything workflow — and it contains a lot of useful information. If you take this information and give it to AI and say give me a kind of what was the data and what was that in this you can create a lot of cool things from this. — That's what we're next. — Amazing. Now you can also see this as almost creating your own AI it buddy that can when errors happen it becomes aware. It can see all the data. It knows what happened with the execution data. And then he said, "Great. Can now can you update my GitHub repo with the appropriate NAN workflow with the fixes in place and then redeploy that? " And that allows you to not only monitor the systems, but then get all the data needed to AI that it could then update your workflows based on all the execution data because now it has all the context needed. So now we are pretty much ready like we're halfway done with what we wanted to achieve initially planned on this excali raw. So if we quickly have a visit so we spun off our instance whenever a workflow is executed it stores it inside superbase. Now what we are going to do is open replet and then build a UI based on this data. Shall we? — Let's do it. So I'm going to go to my riplets instance and create a new app.

And this new app will have certain features that I would like to describe here. So I'm going to quickly type it. Can you create a simple NexJS app with a superb base integration that would pull data from the table? see the schema below and create a simple dashboard that shows the total executions and the failures on a chart and also create simple table using ag grid. For those of you who do not know what's ag grid, ag grid is coming from mostly from angular but it's also in the react and nex. js JS world. It's a great grid application like a package that has all of these filters off the shelf. So that's really amazing npm package and every week it has 1. 6 million downloads. So pretty cool. So I'm going to use ag grid. What else do I have in mind for this? I think that's pretty much it. I have a few other ideas and I will do it as soon as uh ripplet spin spins this thing off uh and then we'll see. Please give it an enterprise look and feel use there is a very nice anything is my new react I used to be I'm a full stack engineer for 17 years some nine years or 10 years of that I was a react engineer full stack JavaScript nojs but react mostly front end so there's a very nice UI library called next UI it used to be called next UI but now it's Eero UI it's really amazing like beautiful UI so I'm going to instruct use hero UI previously called next UI since it's like I really anything's UX and this is very amazing if you know shaden so shaden is another very amazing one but this one is really cool I will give a quick comparison of the checkbox box for example there are small little nuances in this one so you see if you click the checkbox animation this is not possible with other libraries like shenm for example okay so I think pretty much done let's spin off the project with replet and see where it takes us meanwhile this is producing you can ask me any questions — sure fantastic nice so I've connected replet to the back end of n before to be able send data through. And so it's an amazing front-end interface for this. One of the things that I know that I've done before in terms of tracking usage is to be able to track the ability of if a user goes in the system to see how are they active, how often do they go in. Are there any things that you use in terms of replet data of users actually going in and using the system at all? — I went super deep on cloud code, Gemini CLI, cursor and so on. And as far as the kind of like replet and those other coding platforms are concerned, it's the same. — I was curious. Yeah, I do replet in terms of you can access the database, but you're connected directly to Superbase, which is even better to set up system the infrastructure. — So, let me see what input this needs. So, it is saying it's waiting for my input. I don't see mention schema, but I don't ah I see I forgot to copy paste the schema. This is so cool, man. replet is getting much better than I used to use it. So I'm going to bring this schema that I had and paste it here so that it has the schema access. There is a very common mistake that a lot of AI agent developments do. They ask for your anon key for your superbase an key. But this is where this traditional vibe coding like fails and then you say okay if you're a software engineer you start building with AI and you know all the security things in mind and then you can catch these kind of little things as and when they come in front of you. This is one of those pitfalls. So this is the public key and on key the way that particularly this use case should do all the data should be fetched by the back end of the Nex. js basically by the routes the NodeJS side of it and then the data should be delivered from the back end to the front end. So we need the secret key not this one. So I'm going to instruct replet to fix that little security thing and let's see what it says. I would like to use the superb basease data fetching hence the nextjs backend side of things inside the route and then the data flows to the front end so that my superbase can be protected. Now let's see. Yeah, that is the one thing is when we're trusting AI to do so much, you don't know about security protocols and there's places that can people can do prompt injections or other things to be able to get access to secret data. — That's a great security practice. So call app and express back in I don't know not express nextjs expresses the older NodeJS library. Yeah, — thanks. Great everyone. [snorts] Before this thing happened, I was so excited about this security thing coming in front of us and I jumped over this thing quickly. — There's a ton of power in Vive coding or get some stuff up and running, but there is some issues with there are security issues to be aware of in terms of it being susceptible to prompt injections if it's asking you for give me the passwords inside of the system. So, those are things to be mindful of. I watched a person give a talk at the AI engineering world fair in San Francisco where he talked about how he hacked I think 19 AI startups at Y Combinator by being able to use different tactics of figuring out where data is being passed through and then being able to whether take over someone's GitHub repo or [clears throat] it's it was pretty wild on those things. — I'm wondering if that guy also vibe coded the attacking scripts himself. It was a really interesting talk and some things to be aware of. I'm sure at some point we'll have a data security AI system to help review these things but

not quite there yet. The one thing that I really liked after visiting replet after eight nine months is that the pipelines of the deployment they have like in the deployment pipeline there's one security layer that they check all the security things in between and any is like in a totally different corner in a very safe zone and you can go put a lot of guard rails and governance layers around NA10 to make sure that nothing is insecure once NA10 goes to production. But when it comes to vibe coding like these IDEs like replet and cursor and so on, it really goes wild and putting guard rails can be really complicated and I think replet guys have somehow solved that as soon as this front end is up and running. I'll show you what I mean. When you publish when you click this button and publish a web application for example, I haven't tried the mobile application feature of it yet. Have you tried coding with Replet using the mobile app? — Not yet. No. — It's amazing. I was working at the coffee shop. I had to go to another location. Ran out of time, but I still want to keep rolling. So, I opened up the app on my phone and I was as I was going, I was just speaking into it, telling it what to do and fix and update. And you basically had this replet terminal, if you want to call it that, chat window that allows you to speak into it, and it would make the updates. It could push it and do everything. It was wild driving. — Yes. Exactly. So basically you're talking about this and when I open this tab I saw shell I said damn I know this it gave me a nostalgic memory because I've been with these terminals forever and then I said I know this from somewhere exactly where I don't know and then I went to my Gmail let me pull that email for you and guess when was the first time I opened riplet just guess the number which year was 2020. Was it four? — 2024. — Yeah. — Let me then bring it here. 2024. It was — said June 2024. — 2nd of April. 2nd of April 2018. — Oh wow. — So I was teaching in a coding school. Look at this. It was replet. And there are some other emails right away from Look at this. I'm Jed Masad, co-founder and CEO. This is the version that I used to use for my students. I was teaching web development and this is 15th of May 2019. This is crazy times. Meanwhile, the thing is running like how long it took me to build something like I call it VIP kidding. I built it for my son. He is 7 years old and he talks to this microphone and he builds literally web applications, games that you can use and then play those games like car games. — Yeah. My son's not quite there yet. He's only barely even one. But hey, you can only play the games you make, right? — Exactly. And it took me like literally I don't know like five or six prompts to make this. And then it's in 99 languages. It's just a wrapper around clot code and it puts some files together and it blew my mind what's possible today. And look at this. We can build replet or bolt like apps. Obviously, they're like I don't know thousands of years ahead of this what we built. But yeah, you get the point. Okay, so our front end is ready. Let's see. There are some execution errors. I think I know what's the problem. It's still getting the anomon. It didn't get the files correctly done. So, I'm going to put those files in the right place. And then we should be able to get it up and running. I'll just give it because they have a UI and then it will ask me. It's better to do it that way. — Yeah, because there's a couple ways you can do it. You want to put these in as environment variables. it will put it as environment variables but um I will make sure that we don't record some data here let me see oh it's not finished it's still thinking and so I'm going to ask it to ask me so that it's also context aware this is very important when you're vibe coding so that you give like instructions and then go with the vibe and then keep building the context of the chat — if I go and do things separately or with the agent or through the agent then the agent's memory keeps building upon that activity that I do and then it knows what to do next. So I'm expecting to have an ENV file here somewhere — or maybe on the No, it has to be on the server side. So it's not there yet. It hasn't arrived in that location to create the environment. And also it has like these replet key vault stores. So it shows up also here. Really well done. Like replet guys really found out how to do these things very well. Talk to me about the use case of who would find this to be useful and where would they use it? How would they integrate it for these type of systems — for this particular system? Yeah, I would like to give that answer a little bit in detail. — Trying not to give away any secret keys here. — Yeah. So the use case of rilet is pretty much to this for deployed engineers going super fast and this citizen developers trying to build things fast in environments where time really is of essence. What I'm thinking about with this is if I'm running the IT department for this company and then I get note, I'd love to have it if an execution errors out for it to be able to analyze that execution, see what happens, have a recommended solution and then be able to then update with that new solution. I know we have now published versions in this new version of V2 with NAN. We it's not it's no longer just, you know, not active. We actually have published versions of these in ends which I would actually really love to be able to not only I don't know how many people know this too with the executions you can also label and mark them as this is execution this is works well I can tag it as this works I'd love to see as code fixed of this execution and then be able to push that to my GitHub and then bring that in to my in workflow so I can have the AI attempt to try to solve problem — yeah and every one of these version S3. Exactly. Like the execution context keeps billing and all the published versions — are also traced. So, by the way, we just crossed 170,000,000. — That works. I know. I need to get some cool. — Yeah. So, yeah, absolutely. So, the different states I'm not sure if the hooks feature the hooks that we did. — Oh, yeah. this hooks do have a state in here that's called let's say publish pre-publish and so on if yes I need to check the documentation and so on then we get everything in here so perhaps it is already therefore the code completion could pick it but I need to double check that with the documentation of anything — but just to give a final thought on ripplets and why do I think this especially in the enterprises will of its space is this publish feature. So when I'm publishing, look what happens. It goes through a full-fledged pipeline. Not only that, it is an full integrated system, but it also, you see here, it has a security scanner. Right now, it's doing all the security checks that usually vibe coders can foresee like the stuff that we did during this session. and then the build and everything at the side. And then another thing is this shell that I was talking about like literally this is a simple Linux environment. So you can do almost everything in here. So you can see here it doesn't have a git. I want to put a git repository to this. Actually I want to put this git repository here so that we can share this also with the community. So I can create a GitHub and I'm going to make it public call it NA10 dashboard analytics UI and create. So it just creates it and then pushes everything in there. But what's so important and interesting here if I go to the for example the server you can see everything as you see is literally those TypeScript files and so on. So if I check this it's all in my browser. So the vision that Replet started in 2019 when I was using Replet and teaching my students web development, that vision is still alive. It's a browserbased code editor, a fully integrated code editor that's not only in the browser but also now in the mobile phone so that you can walk around and build things as you're talking about earlier. And with all these tooling all together in one place, uh it's amazing. This is a tool that I think is going to have a lot of cool future builders born from this. — One of the reasons why people love end so much is the optics being able to visually see code and data flowing through a system and then realizing you can nothing's really abstracted out versus comparing to other types of automation platforms. You can actually copy and paste and get the JSON all ready to go. And if you do that and there's nothing abstracted out, we're then able to not only take a look at it and have more clarity and certainty, but we're also able to then feed that to AI. So again, what I'd love is the iteration on this at some point, your own IT agent that then goes in and then be a and is able not only diagnose, but recommend solutions and then impose those solutions onto the workflows. And so you have this self-learning model inside of here. I know we can use cloud code and other things to teach its skills in it in skills and other things like that. So I could see this be a self-learning self-evolving system over time with enough context enough data. share my final thoughts about the future being like the software engineer 24 years non-stop building software not like that I did 24 years ago and then I stopped never the always at the front forefront of technology master's degree in AI 8 n years of working in the field and I see these things and I work with these web coding tools I feel that a lot of people are scared if they not using it and they go after these vibe coding memes and they try to not touch it or say bad things about it or do all those memes but if you're not using it you're leaving like I don't know 10x 30x 50x productivity on the table so that kind of a speed and agility brings you a lot closer to proof of value and time to market time to value. So that's directly proportionate proportion to the value you create, right? It's something that it's directly money like the companies will make a lot of money if you become 20x efficient developer and that's the forward deployed engineers and I think all these software engineers in the world should become viers. Memes aside, — yeah, there is a thing that people want to poo — elements [snorts] that aren't purists. I have a friend of mine, he used to code for Doom and Quake and he just wants to be in the terminal. Doesn't want to touch anything else. That's all he wants to live. That's what he wants to play. He's happy. Just wants to hang out there. It's cool. It's great. But the thing is at the end of the day, everybody wants value. The reason why pros and cons to American society like fast food, processed food is cheap and it fills you up. Not nutritionally dense, not a lot of other things, but in terms of I'm hungry, feed me up at low prices, it served a need. The same thing is true is that in whether you're traditionally coding and you're writing in binary or you're vibe coding a solution at the end of the day, people just want results. They want to work. They want to function. They want to be predictable. They want optics. They want to know. And the more that you can expose the systems, show them this is what's going on the back end. This is the error. This is how we're going to fix the error. And instead of you waiting weeks for something to get deployed, we can give you your own AI it agent. I'm going to keep hing on that cuz I think it's a fun idea to be able to solve these problems for you. They don't

care. They just want the value. And so this is the same thing. It's like it's you got to be willing to say, are you willing to adopt to the times or not? And that's fine. — Look at this. I as an NAN ambassador and as a co-founder and CTO of an NAN expert company, we are delivering these NAT workshops around the world. Only in 3 months, we delivered 23 workshops in North America and in Europe. And I personally trained over 8,000 people in Ethan uh in the past 8 n months, 3,800 of them in person. And one thing they kept asking me is that like where is it going? Am I losing my job? This kind of like McDonald's or drive-thru kind of software development era is scaring a lot of people. But n being at the forefront of this backend world of things and repllet and other coding platforms being in the front end part of it will produce a lot of developers and a lot of new so type of software developers are AI native software engineers and I think the world is at the deficit of building software. We'll build more software. We'll make more businesses more efficient and companies will become more profitable with less employees and pay more taxes. The governments will become more profitable and more income tax and then the whole society will go up. So the GDP of the whole world will go up. So we should not be worried about that. and the UBI universal basic income and that utopian future of AI is pretty much plausible. I think we should appreciate this especially the message to software developers who are not trying vibe coding. I think after watching this video they should give it a try. — Yeah. One of the quotes I got from actually he was the head of the Stanford human interaction lab. Walter Greenleaf once told me on a podcast the future is here it's just not evenly distributed and right now you can see the glimpse like same thing like cities when the sun sets not all the lights turn on at once but eventually they all turn on. The same thing is happening. So anybody that's seen this, they're seeing really the edge of what is going on in terms of AI engineering, development, deployment, right? From using Replet to Subbabase to Nad. When you stack these things all together, you can produce value, which the value on this would be optics and speed to resolving any issues. There's nothing more frustrating than having a broken system. Maybe it's a could be some sort of email outreach system or whatever in workflow you have not knowing about it and having it have a whole bunch of errors going on for a week or two weeks and have no clue that's happening. So optics clarity of what's going on is value and if you just keep looking how can I hone my skills and my knowledge with the latest and greatest tools to provide value to these businesses, you will find yourself gainfully employed. — Absolutely. You I think the world needs way more builders than we think. — Sure. If people want to get access to this, get up and running with this, use this system, what do they do? — So the very first thing they need to do is get two GitHub repositories from my So if you go to amslier. com and then you click on this GitHub icon and then you go to repositories

you need to download literally these two GitHub repositories. This is your repository that you need to put on your local machine or on your hostinger or whatever server where you're running your anything instance and this is your riplet or your front end the UI just not UI it's a full stack it has a back end as well but you get the point and then after running that one you need to put that on Versal or wherever you host your Nex. js applications and then you put your environment variables and in the two files as described earlier and then you should be good to go and have a nice little dashboard like this. But you can also modify it and add more things and add in more alerting and notifications and filters and so on. And also this table can have like natural fil I mean off-the-shelf filters that you can pull your data as you need it. So pretty cool project super fun and there's so much more you can do with it. So anybody that is listening to this that ends up taking it downloaded making a whether it's a Verscella app or you clone the replet or whatever you're doing with the systems. If you make it, you build it, you launch it, we'd love to hear from you. So wherever you're watching this, if you do that, comment down below. But we want to know that you actually did it. You built it and you got value from this podcast. Amal, thank you so much for being on the show. I really appreciate you being here. Is there anything else you'd like to let people know about? — Keep building and keep that hope for the utopian AI future. Don't think about the dystopian future. — Beautiful. All right. Keep building out there. Have a blessed and wonderful day everybody. Much love and we will see you all on the other side. Bye now. —