Open Source AI In 17 Minutes

I learned all about open source AI for you. So here's the cliffnotes version to save you the hours and hours that I have spent digging into this topic and building with open source models and frameworks. So in this video I'm going to explain what is open source AI and why you should care about it. The open source AI stack if you want to build things like agents and of course actually show you how to run these models and build with them both with no code and code. As per usual, it is not enough for you just to listen to me talk about stuff. So in this video there will be little assessments throughout this video. So pay attention. All right, let's go. A portion of this video is sponsored by HubSpot. Let's start off by first defining open source AI. Open

source AI refers to AI systems where some or all of the core components are publicly available. Typically including things like model architecture, model weights, training code or inference code, a license that allows use, modification, and redistribution. This is in contrast to closed source AI which refers to AI systems where the models weights and training process are all proprietary and users can only access them via API web apps or enterprise platforms. Some common examples of closed source AI include GPT models from OpenAI, Claude from Anthropic, Gemini from Google and Grock from X. Now, since 2022, Closer AI has just been dominating the field primarily just because it's so much better than its open source counterparts until January 2025 when DeepSync R1 came out. This was the first time that an open source model was be able to compete on par with the best closed source models at that time, which was a really big deal because it opened the floodgates for more open- source AI to start showing up on the leaderboards. Open source AI has huge advantages. You're able to have full control. You can run it wherever you want on prem, edge, private cloud. It's customizable. You can fine-tune it, modify the architecture, add guard rails that you like. And there's no vendor lock in like closed source AI. If you use those, usually you're locked into their entire ecosystem, which is more restrictive and much more expensive. And speaking of cost, cost is one of the biggest advantages of open source AI. It is so much cheaper. Open source AI is also auditable, which is really important for regulated industries like healthcare or finance. and finally allows for innovation because developers from all over the world can contribute towards these open source models. So yeah, it became a no-brainer for a lot of companies to switch over to open source. What is also interesting is that the top tier opensource models are Chinese models. You can see from this graph here that the US was leading in the number of downloads of open source models up until around June July of 2025 in which there was a flip and Chinese models really skyrocketed. A16Z, which is a big American venture capital firm, says that the odds these days are that their startups are using AI models made in China. I'd say 80% chance they are using a Chinese open source model, says Martin Casado, a partner at A16Z. And probably because of these geopolitical tensions, we've seen that in the past few months, the Western world has also been responding by investing more into open-source as well. So, as a whole for the consumer, like you and I probably, this is a really great thing because everything is becoming more open source. Oh, and just a side note because I did mention like Chinese open source models and different open source models in a previous video and people were talking about being concerned that, you know, we'll be running things on Chinese servers. The whole point of open source is that you don't need to be running on anybody's servers. You can download them locally. You can host them the way that you want to. So, you do have the data privacy there. Anyways, of course, I'm not saying that open source models are completely perfect, okay? Open source models do also have their cons. They have higher setup complexity. Unlike close- models where you can just literally click a link and then just start doing whatever you want with them, there are a few more steps that you need to do in order to access them. Hardware is also a big thing. Running AI models is a pretty resource inensive thing. So, you do need to have good enough hardware to support this. There's generally speaking four different sizes of models. So, based on the hardware that you have, you're limited to the models that you can run. I'll put on screen now some of the requirements. Open source models also do have weaker capabilities straight out of the box because they're not equipped with all the bells and whistles that you might be used to. like if you're directly using a closed source model like claude or GPT and finally because nobody is managing things for you and accessing your data you need to do this yourself so you do need to manage things like security scalability and uptime with that being said though remember the last benefit of open source which is innovation because open source developers are all able to work together to improve things these drawbacks are also rapidly shrinking these models are getting better and better out of the box with technologies like inference quantization making these models smaller so now you're able to run a lot of these models on your own personal computer. Open source developers are also building frameworks and packages to make the setups easier and maintenance and scaling easier as well. So yeah, TLDDR, open source AI is very much exploding right now and this is a great time for you to start exploring and building with these models. Like literally, you could not have done this just like a few months ago unless you had like commercialrade hardware and professional coding skills. Time for our first little quiz. Please answer the questions displayed on screen

right now and put them into the comments. All right, let's now talk about the open- source AI stack.

So, first, just to make sure we're on the same page here, what I mean by open source AI stack is the collection of technology and frameworks that you would need in order to use open source AI and build things with them like apps or agents, like a personal assistant AI agent that can do things like book your flights for you, manage your calendars, or like an AI dating app where the app is using AI to match people's profiles. Or it could be like I guess AI's dating each other. I don't know. the sky's is your limit. But yes, let's talk about the technology and tools that you need in order to build these things. So the most important thing of the open source AI stack is of course the models. As of the filming of this video in February 2026, some of the leading open source models to keep an eye out on are Kimmy models from Moonshot AI, GLM models from Japu AI rebranded now as Z. AI, and Hunya models from Tencent specifically for image processing. Of course, I don't know which ones are going to be the leading open source model by the time that you see this video. These are going to keep changing, but it doesn't really matter because the way you use them is still going to be the same. You can see their current rankings, try them out, and even compare them if you want by going on websites like LM Stats or Arena. But if you actually want to use them locally and build with them, you need to use what is called a model manager. The most popular one is called OAM. Just go on their website, download the software, install it, download the models you want to try out, and then you can just run them. That wasn't too hard, was it? So amazing. Now you actually have your models downloaded locally, like they're running on your actual computer locally. This very powerful AI model. That's just like mind-blowing to me. But we're just scratching the surface here. If you really want to start experiencing the power of these open source models, then you need to start giving them access to things like tools and memories and knowledge. H are there any long-term watchers of this channel here? Thank you very much for supporting this channel. What does this sound like to you? Yes, agents. If you really want to experience the power and the benefits of these open source models which we talked about earlier then the best way to do that is to start building agentic systems. Now the point that I'm trying to make here is that open source AI is not like a different species to closed source AI and open source agents is not a different species from closed source agents. This, by the way, is exactly why in all of my videos, our workshops, our agents boot camp, we focus on the fundamentals and the principles, not any like specific tools or technologies. Because you can take everything that you've learned building AI agents previously and apply them, but just using open source AI and the open source stack. So, I'm not going to go into extensive details about how to build agents. Now, I'll link a video over here also put some in the descriptions. But very briefly, if you remember, the components of an AI agent is models, tools, knowledge and memory, audio and speech, guard rails, and orchestration. And for building open source agents, it's still the same components except the models that you're using are going to be open source models and you do need a model manager type software like Olama, but everything else it's the same. I'm going to put on screen now some open source compatible tools and technologies that you can use to build your AI agents. And by the way, some of the most popular agentic orchestration tools and frameworks can also be used for open source models too like NA10 or if you're a developer, OpenAI's agents SDK and Google's ADK actually support open source models too. And there's also open source first agent infrastructure like langraph, llama etc etc. Oh before moving on to the next section I do also want to make a note that open source AI models and infrastructure is particularly dominating in the AI coding arena. There are a lot of open source coding tools available now on the market. Our company Lonely Octopus just a couple weeks back we actually hosted an event in Hong Kong all about open source AI where people were able to vibe code really cool projects in under an hour. the three winners from the event. The first one is Git Snake where you can learn Git by playing Snake. The second is Palmer Reef. That's a productivity tool that can turn focus into a visual experience and Asian life where you celebrate Asian life stuff. I'm going to put on screen now some open source AI coding tools uh that you can check out if you want. Most people can generate ideas with AI but struggle to turn them into actual deliverables like tailored reports, company specific analyses, and presentations that are ready to share with your team. Now, HubSpot just released a new free resource called the Claude Co-work Stack 12 advanced prompts to replace a week of work. And it's designed to close this gap. As his name suggests, it includes some really great prompts, including one that analyzes your content performance for the past months to find out what's working and why for your specific audience. There's one to generate a competitive intelligence report by analyzing competitors based on public information and documents that you've collected to get insights. And my favorite, one to do your weekly ops prep. Ops is like the bane of my existence. This will let you start your week with a clear list of your priorities flagged, prep work identified, and calendar synced to what matters. Each of these prompts is built to work with your files, your data, and your context so that you can get deliverables that you can actually use. All you got to do is just copy paste to get started. You can grab the free resource at the link in the description. Thank you so much HubSpot Media for providing us with free AI resources to help us all level up. Now, back to the video. Okay, great. Let's now move on to tutorials and demo time both with code and no code. Let's start off with no

code using Olama and NA10 to build an open- source agentic workflow. All right, demo time. So here is a simple agentic workflow that's able to take financial statements on my local machine. Like literally here my financial statements file. There are these three credit card statements here. This is something that I would never upload into close source AI, right? Like because of privacy reasons. So they're all local. And it would take these files, extract the files, and then pass it over to the AI agent, which is using a open source model, the Quen 38B model specifically, and is able to analyze my spendings and give me some tips. So, let's run this [clears throat] and some results that we have. So, we see that the AI agent ran all three of the statements. We can see that here's a summary of the monthly spending categorization for July and August. Uh, we spent a lot of money on digital services. That's like our biggest expense, as well as traveling. And same with August as well, mostly digital services as well as some traveling. It suggests that we can cut some non-essential digital subscriptions, maybe cancel some unused software like Heroku, Medium or Slack or downgraded cheaper plans. I think that's very fair. It could save up to 2253 annually. Seems like we do need to review our subscriptions and stuff like optimizing travel costs. So, some key takeaways and then also gives us some action items to work on. And it does this for all the other statements as well. What is really amazing about this workflow is that it is all hosted locally. So nothing is leaving this machine. No privacy concerns and it is completely free. That's crazy. Of course, there are a lot of ways that you can improve this agent. For example, giving it more information from other financial documents like bank statements or investments as well. So it has a better picture. You can have it automatically pull that um from your bank and other places as well all locally and then it can run automatically. You can also have a dashboard. It's a lot prettier for you to look at. There's so many other things. So if you also want to start building local agents like this, you need to download Oola Lama and then you also need to download NA10. Olama manages the open source AI models for you on your local computer. So this like the model part of the agent and then you can also get NA10 locally as well which helps build out the other parts of the agents by giving it stuff like memories and tools and orchestrating all of them together. I recommend going with the self-hosted Docker approach and you can just follow the instructions. If you want to learn more about how to use NA10 to build agents, I do have an entire video about that which I will link over here. So, I'm not going to go into too much more detail about that here. So, yes, you can download a Lama and get NA10 and then combine them together. Alternatively, which I think is an even easier approach if you're okay downloading um and running a little bit of code, like you don't actually have to write the code, just run it. N does have this self-hosted AI starter kit. So, you can just come here, download it, and then go through the instructions based upon the kind of hardware that you have. It comes with the self-hosted N8N. It also comes with O Lama so you don't have to install these things separately. It also comes with quadrant as a high performance vector store really useful as you start building more complex workflows and also Postgress SQL which can handle large amounts of data safely as well. So I'm not going to go into way too much detail either about how all of these fit together but it does come in really handy for a lot of different AI workflows. This is what I did by the way. So after you do that, you will see something like this where then you can click create your workflow and you can start creating stuff if you want or if you want to run and build on top of the financial statement analyzer I just showed you. You can just click here and then import from file. I will link the file for the workflow in the description. Might need to fiddle around a little bit setting up the models, downloading things, configuring stuff, but yes, you will end up with this a free and private local agent. Great. Let's now move on to the code version.

We'll be using Python with Olama and the OpenAI agents SDK. Demo time for the code version. So, here's an email agent workflow. So, I'm going to run this now. Run this agent for new emails. This is warp, by the way. Um, it's how I'm using it to run this agent. It's like a AI coding agent. Okay, so it's starting email agent workflow fetching up to five unread emails. Found some emails — later. — Okay, cool. So, it finished everything and we can see that there is a Lonely Octopus interview email where a draft is created. We'll go check that out in just a bit. And then these are some other ones that it ignored because it did not require response. So, let us check it out. So, here is the email that's telling me, oh, I'm pleased to inform you successfully pass the initial interview stage for the software engineering position at Lonely Octopus. We'd like to invite you to our final round interview. Please reply by Wednesday, end of day with your availability. So the agent drafted this reply. So dear Octopus team, thank you for the invitation. I'm available on Tuesday, Wednesday between blah blah. Please let me know what time works best for your team. Looking for your confirmation. So I didn't send this yet because I actually have to go in and edit it so that it can actually send this response cuz I don't want to just to send it directly. That's there. And then I think it's mostly in the promotions here. Yeah, like it read the promotions and it decided like, oh, we don't need to respond to that. So in this case, I didn't have any emails that got flagged, but I want to show you what happens when it does get flagged. So here is an example from like last night. It's about like a Rev like application request situation. So it got flagged for human and it says the email appears to be automated job listing notification with embedded HTML or shortened link. So it suggests that this might be important and it got flagged because you don't know if it's a potential security risk or if it's like an actual thing or not. So it would send me an email notification on this. This is really good for personal use. If you get a lot of emails and especially for business, for the company, it is really useful because we get a lot of emails like a lot of requests that are coming in for collaboration, people asking questions about our programs and a lot of other things as well. So, this is a great way to screen these emails, draft the replies, and it makes our workflow so much easier. That's why we're able to handle that much volume even though our team size is actually really small. We only have five people. This is only doable because of open source AI. Firstly, I would never send my emails to some company in which they can use that data to do like who knows what, right? like that is definitely not going to happen. And even if I worked out some like privacy workaround, it would cost me so much money because I would be constantly monitoring these emails, categorizing them, and drafting replies to this. But guess what? All of this is completely free because it's hosted locally. Crazy, right? So, just in case you are interested, I'm going to put on screen now how this multi-agentic system works. There's three different sub agents that are there and I built it with OpenAI's agents SDK with Olama with the Quen 38B model which of course you can switch out to other models very easily as well. And I'm going to put on screen also the prompt that I used to give to warp where I basically just told it to make this multi- aent system based upon the documentation from OpenAI's agents SDK and was able to like 95% oneshot it. This is not an AI coding tutorial so I'm not going to go into more detail about this here, but if you're interested in a video like that, let me know in the comments. I'm also going to put the code uh linked in the description if you want to play around with it. There is so much more that I can go on about right now, but there's no way I can fit that into this video. I'm going to put other videos and live streams that I've done in the description if you want to go more in depth about building agents. And also, we do have a 28-day boot camp as well that goes a lot more in depth. You have a clear learning path. There's community and there's mentor support for everything if you prefer a little bit more structure in your learning. So, if you're interested, you can check it out at this link over here, also linked in description. Pretty crazy. We sell out within 24 hours usually um just to the wait list because we only have 100 spots available per cohort cuz I want to make sure everybody gets the best experience possible from our human team. Amazing. Great. You now know the fundamentals of open source AI, the open source AI stack, and how to build open-source AI agents both with code and no code. I understand everything. You also have a lot of tools and resources that you can start exploring too as you continue building. I really think open source as has so much promise and it might just be the push that's going to really help companies and individuals incorporate AI into their workflows. A lot of the hesitations we have about privacy and cost, these are not like solved by any means, but I do think open source AI provides a big step forward. Now, finally, here is the last little assessment. Please answer these

questions and put your answers into the comments to make sure that you retain all of this information that we just talked about. Thank you so much for watching until the end of this video and I will see you in the next video or live stream.