# STATURDAY CHAT LIVE with Brandon Foltz, Ep. 64

## Метаданные

- **Канал:** Brandon Foltz
- **YouTube:** https://www.youtube.com/watch?v=wxAhk46xE64
- **Дата:** 03.03.2024
- **Длительность:** 52:12
- **Просмотры:** 515
- **Источник:** https://ekstraktznaniy.ru/video/52869

## Описание

Very informal, casual chat and hang out! If you have a question, you can submit it by messaging me here or on my website. Stop by for a bit as I talk about things I am working on, answer questions, and have you tell me how I can serve you better. And, of course, I want to check in on everyone and see how you are doing. See you... STATURDAY CHAT LIVE!

#statistics #datascience #machinelearning #ai #qa

## Транскрипт

### Segment 1 (00:00 - 05:00) []

hello everybody and welcome to the next edition of Saturday chat live here on Saturday March 2nd 2024 I hope you all are doing well wherever you happen to be in the world and on that note please let me know where you're watching from I always love knowing where people are joining from I appreciate you know different people in different time zones that really make the effort to come on the stream and just chat for a few minutes and uh I'm just interested in sort of geography in the world so let me know where you're watching from so I really enjoy that so I hope you're doing well it's a uh wet spring day here in the Midwest of the United States but it's warming up so I'm glad that spring is just around the corner here because I love sitting outside and working out there and um you know just enjoying the Sun so I appreciate uh I appreciate all you being here on a Saturday I know you have a lot of things to do that you could be other places you could be and of course I have a lot of things to do um as well so yeah so the way this works if you are new to the stream I see we have a couple people here that are new Phil good morning first time to be live in your chat thank you Phil for stopping by I really appre appreciate you stopping by very and Boise excellent I've never been to ID Idaho but I know it's pretty beautiful out there I like the west and I hope to get out there of the long list of places I hope to get to at some point but Phil thanks for stopping by so the way the live stream works is I go for 30 minutes or sometimes an hour depends on how the chat goes depends on what you all want to talk about and you know we don't beat the dead horse on this live stream to make it longer we will end when it naturally ends depending on what you want to talk about so that's kind of the way it works you can put your comments or questions in the chat wherever you have to be watching from I will do my best to answer them live if I can but I won't give you false information if I can't you know reliably answer the question here just off the top of my head you can always contact me on LinkedIn is usually the best way or through my website's contact form which I can show you if you want to know kind of where that's at but I will do my best to answer whatever questions you might have you know live here as we go so we have um viewer from Uganda excellent thanks for stopping by that's awesome to have um to have you here I appreciate you giving some of your time to the live stream today so a few house cleaning things oh there he is he's back Piper trip is back cheers from Denver you hope you're doing well it's good to see you love it when uh when have returning viewers and people that maybe have are busy for a little while and they come back after a little bit which is completely fine this is I do this for you it's not really for me it's for you so you always come by the live stream when you can so a few uh updates on where things are with the channel and what I'm doing and so on and so forth so of course I'm continuing to release videos when I have time so as of right now you know I'm busy at my 9 to-5 job which is going very well I'll have more on more info on that probably next live stream I can uh talk more about that freely that's going pretty well I have another side job and then YouTube so I do YouTube when I have the time because right now I have to do whatever pays the bills as I'm sure you can understand you know this is not the time with inflation and everything to be doing a whole lot of free work unfortunately I wish I could but I do it when I can I have a lot of videos lined up I know what I want to do I have some scripts written it's just a matter of finding the time to do it all because I do it all myself as well you know I'm one I'm a oneman show so um so continuing to release videos on the channel and we'll to see

### Segment 2 (05:00 - 10:00) [5:00]

when those are released I do them when I can that's just the nature of the Beast of making free content and then these live streams depending on a few factors may come to an end in April so the I went over this last stream but the last yeah software I used to do this streamyard is extremely expensive it's like almost it's between $4 and $500 a year and right now YouTube is not paying out enough to justify all that kind of stuff so I'm actually losing money on the channel so um I might use OBS to do some live streams but I'll have to figure that all that out so I did it I did that at one time but streamy makes it so much easier to do all the graphics for you and all the comments for you it just makes it a whole lot easier so I'm not sure how much longer the live streams will go um could be ending in April just depends on some circumstances and finances and stuff like that but I love talking to you all you know but it has to be sort of the juice has to be worth the squeeze as the saying goes so um hey Brandon in this session my first live attendance do we discuss statistics here we can if there is a if there's a stats question that I can answer off the top of my head I will do my best if I cannot then I will defer to you reaching out to me on LinkedIn or via my website for maybe a little further discussion as long as it's a quick question that you know it's in my memory I'm happy to do my best to answer that or if I don't know I will tell you I don't know you the thing about these thing about channels like mine is you don't keep all the stuff in the front of your head all the time because you can't but I do my best to uh to answer whatever questions you might have of course so Piper yep got busy here I absolutely I can feel that you know too so end of January some of you may not know end of January my grandmother passed away so we've been busy with uh you leading up to that she was she had been ill for a while and in poor health but it's always you can never be fully prepared for something like that so we're you kind of going through that and then my father my dad had surgery a few days ago on his leg so it's busy here we're all just trying to like keep our bodies put together right now so yeah busy as well I've been traveling doing some traveling for work and I'll be coming I'll be doing some traveling for work in March I'll be done in Texas I will be in the four for Worth Dallas Fort Worth area in March so for a customer event for you know for the company I work for looking forward to that so yeah lots of stuff coming up as well busy busy as we all are so yeah put whatever questions comments you have in the chat and I'll get to those um you know in due time so I do have something I can show you today that I've been doing it's not particularly stats related but it is AI related so I can show you that and maybe you'll find it interesting won't take too long but it the basic idea is I want to quickly show you if you want to how you can run language models on your personal computer that are open source that aren't really connected to the internet or anything you can download these models and it's all your system it's all self-enclosed it's all private and you can download them and play around with them and kind of um see how they work and stuff like that so you know if you if you're not able to pay the $20 a month for uh GPT 4 GPT plus whatever it is or gemini or co-pilot or whatever this is an alternative where you can download these models and put them on your computer and run them locally depending on what kind of computer you have and I'll talk about that um here in a minute when I show you how it's done okay so I'm going to show you how to do that if you want to play around with it which I think everyone probably should I've been preaching on this channel for the past year that everyone should be keeping up to date with all of the AI developments because if not the world would just steamroll you I'm not saying you have to be a star coder you know writing crazy applications and scripts to do this stuff or be a prompt engineer Maestro but staying on top of it keeping up to date with the new features new models that are being released how it's impacting job markets and stuff is very important and the skills I have taught myself over the past year have absolutely positively supercharged my skill set it's a superpower I'm able to do things now that I never thought I'd be able to do because I have ai as an assistant it's just crazy so um so yeah so I'm going to show you how to do this on your local computer if you have the the Hardware to do it

### Segment 3 (10:00 - 15:00) [10:00]

now if you don't you can still probably do a smaller model but it might be really slow but it's theoretically possible okay all right so let me share my screen and I'll show you how kind of how this works now you can keep putting comments or questions in the chat about anything while I do while I show you this and we'll get back to those if it's about what I'm doing then I'll answer them in real time of course but you know you don't have to pause commenting and questions and stuff just because I'm doing showing you this okay so let me go back to my screen here oh before I show you how this uh works just keep in mind uh the channel make sure you're subscribed if you're not I would assume that you are if you are here but another thing share the channel if you don't mind you know there's been a lot of changes in YouTube over the past six to nine months and channels like mine have really flattened out because you YouTube wants to be a streaming service not really a YouTube anymore you know that's how they make their money it's that's their right um so a lot of things in my channel require you all to share content share videos share the Channel with your classmates colleagues whatever um because it's really really slowed down compared to what it was you know probably about nine months ago I think is when the change happened all right so um on that note just real quick if you want to contact me like I said before you can contact me through my website here I got a really nice message from someone the other day um just telling me how thankful they were for the channel it was really nice to get that email for the from this contact form so if you go here you'll just have your name um name email message and then it'll send me the question and then I can respond directly to your question if it's short you know I can't I'm not going to sit and do a dissertation for you but if you have a question or just a comment you know whatever um just put it in here and click send you know so um I've been following your channel when it was 50,000 subscribers yes that was U well one I appreciate that very much I really do it is it means a lot to me because my whole point of doing the channel was just to make a difference to help people and I started the channel at a very low point in my life during the Great Recession that's when I started the channel and it really just took off and it's been a lot of it's been very helpful to people I can tell from the comments that you see on your videos from the comments I get off videos either through Linkedin or through my website that it's been very helpful to people and that's the whole point that's why I do it you know right now I'll be quite honest with you right now on YouTube I'm making a few hundred a month um it used to be like four times that and that's just the change in YouTube and the change in the economy and so on and so forth so I've never really done it for the money I do it to help people so I appreciate your viewership um sanes and I hope you stick around for the next 50,000 um so yeah so here you can also submit your questions here all right so that's I want to make sure that you were aware of that all right so how do you run language model locally on your computer so the way to do that here is through a more room program called LM Studio let me make this a little bigger for you okay so this is called LM studio and what it basically what it is and I'll obviously I'm going to walk you through it it's sort of a program that allows you to interface with different language models that you can download and basically have your own sort of GPT on your computer assuming you have the hardware to do it and I will describe what Hardware you should have here in a minute so as you can see here there are there's a version for Mac windows and Linux that's in beta but I run Windows so of course that's the one I use very simple you just download it and install it I think their whole website is this one page that is it okay but download it and install it and when you do you will have something that looks like this so this might be hard to see I can't I don't think I can um make this bigger yeah I cannot make this any bigger sorry so hopefully you can see most of this how about this I'll just yeah I'll take me off the screen just for a little bit that way it's bigger so the way this works is that in Studio you

### Segment 4 (15:00 - 20:00) [15:00]

have there's five little tabs on the left and I'll walk you through what some of these are in a second then this home screen and what it can do is this home screen tells you what sort of models that you can download and install on your computer so these are open source models by the way that's why this is so cool this is all open source it runs on your machine like I said it's private you could theoretically run it offline you don't have to be on the internet to run these models and you can get like a chat GPT type experience for free for nothing okay so when you look at this home screen it's going to give you some helpful information so you know how to proceed okay so these models here it has the what the model's title is it has how many parameters it is usually so two billion 7 billion three billion then the name of it so here we have this is Google's Gemma that was just released this is a Quinn this is F2 we're going to download that during this demonstration but most importantly it kind of tells you what kind of Hardware you need to run this model and this is where you really have to be careful otherwise you're it's going to be really slow so when it says requires eight plus gigabytes of RAM okay what that's really referring to is not necessarily Ram like CPU RAM on your motherboard or computer but RAM on your GPU your graphics card so for example let me put me back up here for a second so this is my old graphics card which is it's just sitting upstairs because I have a newer one now um but this is my old one okay which I guess technically is for sale if you're looking for a graphics card let me know but this number right here that you're looking at okay this is your vram the reason this is important is because Lang language models run on gpus not really CPUs that's why Nvidia is worth $ 47 trillion doar right now okay so you want to look for this number on your computer or your video card ideally you want to have at least eight gigabytes of RAM on your vram in your GPU to do this you could do it but the model will be excruciatingly slow okay so you need that's what this is the number you're looking for and you want to have at least eight is the ideal all right so what we can do is we could just set we could just download the FI to this is one I wanted to show you this is one by that Microsoft released and what I'm going to do is I'm going to go ahead and go to the search box okay I'm going to clear this now here you can search for models so if I type in 5 2 okay it's going to give me all these different versions of this because they are open source you can't even see can you because they are open source the um there's many versions of them okay so you can pick which one you want but the one I always recommend from and this is coming from hugging face by the way is the bloke so whenever you see the bloke that's the one ideally you probably want to download okay so this is the one where we're actually going to download so I'm going to select that and then over on the right you're going to see some very important information so in this list there are different versions of these models to keep things simple what you want to do is download and install the biggest one you can the bottom of this list okay and ideally it will say full GPU offload possible okay full GPU load possible actually what I'm going to do is I'm going to delete this so we're going to install it okay all right so if it says full GPU offload uh possible that means that LM Studio has looked at your system and said you know what you can run this entire language model on your current GPU all right so basically go as far down the list as you can it might say like partial GPU possible that's up to you if you want to try that but ideally pick the long the one at the bottom of the list that says full GP uh GP load possible all right I'm going to go ahead and click download and it's going to begin downloading all right now these files are huge okay this one's almost three gigs but you can see that my internet is

### Segment 5 (20:00 - 25:00) [20:00]

pretty fast so it'll only take um a couple seconds here but once you download these models they're actually on your computer and they're available for other software so if you've ever heard of ama can connect to these models that you downloaded through LM Studio which is a whole another thing but you only this is a nice repository for any models you can centrally access okay so it's about done and once it's validated we'll go ahead and open it up and give it a whirl all right so it's done now so that was the search box now we have the chat box okay so I'm going to select f 2 the one I just downloaded it's going to load up actually let me do this let me show you this what I'm talking about let's do a new chat this is what I mean by GPU here okay so here you can see this is why this is important okay I want to have all this up and actually I want to show you this before we load the model so you know what's going on over on the right hand side you will see this full GPU offload button okay if you check that it will try to put the model on your entire the entire model on your GPU okay now if you uncheck that then you can put this slider or uh however many um layers of the model you want now you might be asking yourself how do I know how many layers the model has to put on my GPU well the way you can know that is if you go down to Advanced configuration and model inspector uh did they move it where did it go basically there is a setting in here and this says been updated where I can't find it anymore ha I'll see if I I'm not going to sit here and click around this for you and bore you all to death anymore um but basically what you can do is you can find out how many layers the model has and then you can decide how many of those letters layers you want to put on your GPU okay I'll see if I can find it here in a minute but I'm not sure where they moved this to this has been updated a few times since um I've been doing this so I'm not sure okay so anyway I'm going to do full or Max offload you see this Max over here and that's going to put this entire model on my GPU okay now if I do that right now you keep an eye on this so right now my vram is at about 16% usage because I have a 4090 um 16% but if I load 5 2 watch what happens to my um vram once it loads up so you see that sorry I'm having my uh work my post-workout shake you can see how much this very small model how much it took up in my vram on my GPU so I couldn't so I want to stress enough you have to have enough vram to run some of these models but LM studio will help you pick whatever that is all right so we good to go we have a new chat in here we are running this 52 model on locally on the computer okay and now you can begin with whatever you want to talk about so let's say um if I Want To Learn Python for the first time how should I start okay now when I hit enter watch how fast this is because it's all on my GPU look how fast that is okay it's almost instant because it's running locally all right so it's a good way to start pythons is by Tak an online course there are many resources available um it's also important to practice writing code as this will help you develop your skills and understand language better how about if I am

### Segment 6 (25:00 - 30:00) [25:00]

studying machine learning there you go see this is again this is all running locally on my computer and you can do all kinds of stuff you can Branch a conversation so you can take part of conversation and Branch it out into a different sort of thread you can of course copy um the output you can edit the message and rerun it okay of course and of course you can delete it so you see user AI user AI so basically you have your own little GPT for free running on your computer boom okay um what are important python packages for machine learning so numpy pandas M plot lib s kit learn there you go so obvious oh there it is I'm not sure where it where this came from this is what I was looking for before by the way you can't see this but right here it says numb layers so 32 so this 32 is what you would want to have up here in this slider to fully run the model on your GPU if I set this to 16 it would put 16 layers on my GPU and try to run the other 16 on my CPU so that is down here in the model inspector that I couldn't find before this is what I was looking for all right so again this is a way to um to run local llm M or llms locally on your computer assuming you have enough resources on your machine to do it and it's all free all right now I'm going to eject this model and load a bigger one so you can see just how crazy this stuff can get and then we'll do something else and I'll get back to the chat I promise okay all right so you can see my vram went back down to 16% now we're going to go to little bonkers I'm going to load llama 2 okay actually let's do mistel we'll do mistal instruct this is a 7 billion parameter model I'm gonna load that up now watch my vram R boy yeah so when you see that instant jump in vram that was loading this 7 billion parameter model um so my GPU is 24 gigabytes of vram so this is taking up plus everything else half of that all right um but see now when I run now when I rerun that same prompt it's using mistol and I'm going to get a different response okay so popular P python packages for machine learning include CIT learn that tells you what it is tensorflow um carass P torch numpy and on and on okay but again that's taking up you know a crazy amount of memory but again this is my own little GPT on my personal computer that's free that I can use all right and finally I'll show you um you very few people can do this but I can because of my graphics card okay I'm G to load a biggie we I'm going to load Lama 2 13 billion okay which my computer might explode but watch my vram here and I'm also trying to show you the different kinds of output you get based off the size of the model okay so look at so now my vram is up to 80% do you see the difference in each model size and how much resources it takes as you go up so now if I hit regenerate here see the difference so you're going to get a little different output based off the model you're using Okay um and it tends to be more contextual a bit more informational the larger model that you use okay so I'm going to eject that one

### Segment 7 (30:00 - 35:00) [30:00]

and then you can maintain your models where this folder icon is right here if you want to delete a model and make space on your computer you just do that here with this by this folder okay I'm going to go back to the search real fast and let's give Google's new one a spin so we'll download this one real fast um this one looks pretty good let's try this one okay this is a little bit bigger but I'll go ahead and download it while this is downloading I'll get to the chat and take whatever questions you might have okay so while I'm waiting on that to download let's see here um BR what is the best way to monetize teaching I want to use my stat skills to earn some money from teaching as a side hustle your point of view will be helpful well that's kind of what I was getting at before the unfortunate reality right now is that it's very hard to monetize uh teaching because one there are so many people doing it the cost of two the barrier to entry is basically zero three there are many people that are willing to do it for free so it's hard to charge for stuff that other people are doing for free that's one of the reasons I've never had paid courses or anything like that because one I don't want to my stuff to be a financial burden on anyone but the trade-off is I only do it when I can that's the trade-off for having free sort of content but you can try things if you want like um you could do a uh like some people on YouTube also have a patreon where they only post certain videos on their patreon and they teach that way so they'll keep their YouTube main YouTube channel pretty generic and then they will have a patreon where they do more um more value ad type videos for the members only same thing on YouTube I don't have memberships on my channel because I don't want to be obligated to make content all the time but you can actually make videos on YouTube for members only and that's a way to generate a little bit of income through memberships on YouTube then you have options like um udem me which as long as you pass like a few technical sort of tests and stuff you can and create your own courses on udemy challenge with udemy right now is udemy is pretty much dominated by a handful of people who do it full-time and they just flood udemy with courses they're good but it's really hard to compete with people who do that full-time and just have dozens of courses on there that they cross sell and cross-pollinate and stuff it's really hard to just put up one video on there and do it there are other sites like teachable and think giic that you could explore where you can actually you know offer paid courses but it's really hard to do you know if you know I always I'm wearing my uh my educator extraordinaire shirt today because it was the first one I grabbed but it's just it's really hard to make a make money making courses that are paid unless you're somehow able to do it full-time because you have to the volume that's why I've kept my YouTube Channel free over all these years and not put anything behind pay walls I just have the ads on there I take donations I don't really get many donations really anymore unfortunately um and then I sell my companion PDFs which by the way probably won't last much longer either because it costs more to host the site to sell them than I make from them so it's really hard and um the last thing I'll say too and this is not to anyone necessarily in this chat or on my channel whatever but there's also a lot of entitle entitlement out there there's a lot of people who think they are entitled to get all of your best content for free and it's a complete devaluation of your time or disrespect of your time so unfortunately that does exist so you can try different things and see what works but I will tell you that it's a long road it's the long G game okay that's the long game thanks for making stats so simple you are very welcome all right let's go back here and um let's load up our new Google model that we just uh downloaded and see what that's like so put up my Ram here this one shouldn't be too bad because it's only two let's load this up see what this is like

### Segment 8 (35:00 - 40:00) [35:00]

how much would this take see that's not too bad 35% of my vram no problem okay full offload let's see what this does so here's important packages for machine learning numpy Panda py it learn tensor flow P torch M plot lib and Seaborn yeah so this is again running the Google model up here the gimma one all right so I would if you have the ability I would definitely encourage you to down there's no it doesn't it's all free You Know download LM studio if you have a decent graphics card and if you don't know what you have you can look in your um Windows like I'm on windows so I can hit control alt delete I can do my task okay so manager here and I can see um how much dedicated GPU memory I have it's right here it's small 24 gigabytes so you can tell what you have in your computer by using the Windows task manager is control alt delete task manager and it'll tell you what you have to work with okay on a Mac I'm not exactly sure how that works but you can look at your system specs you can figure it out I'm sure okay so you should definitely download LM Studio download a few smaller models start with the smaller ones like um I'd recommend fi so this F 2 is a really good model for its size it's really good for its size okay the Google one is not too bad either that's the new one that's released but once you get you know three billion parameters is probably you know but once you get into seven billion models and 13 billion models you got to have the have the hardware to do it there's no doubt about having the hardware uh to do it all right but again for something that's free and takes two minutes to try just do it okay just do it again this is LM studio um it's also cool for trying out different models because there's no all you have to do is download it so you want to try like a coding model you can try that um if you don't like it just go into your models here and delete it okay last thing I'll say here is that um this is an advanced feature but you can if you take a few minutes to learn how to do it you can run a model that you download on LM Studio you can spin up a local server all right a local server and then you can actually connect to this server running the model locally from your own computer like through a python script if you if you've ever written a um python script that connects to open AIS API or whatever else you can do the same here so you can actually spin up a server here that you can connect with to you can API in locally from other scripts that you're writing or other applications that you're writing so for I give you a perfect example H hell Hell's Bells where's it at um do I have it here so I use um I use streamlit to build um I use streamlet which is a python package that will actually build a front end for you I highly recommend it so what you can do is you build the front end using the streamlit package in Python and then you insert your python sort of a different points in there and it'll build this fullon um web application that you can then run locally or you can run it on render uh do and if you know I have a few minutes left if you want to see that I can show you that it doesn't take very long for me to show you that but my point is you can actually run up spin up a server locally that is running this local language model that you can then connect to through other things which is crazy it's like having your literally your own open AI server running on your home in on your network it's crazy um so I haven't uh

### Segment 9 (40:00 - 45:00) [40:00]

messed with that quite yet because I was using AMA to do some of that stuff but that is possible it is absolutely possible to start a server here and run it and then if you look here this looks exactly like the open AI API call because it is it uses the same thing so the way these models work is it uses is sort of um how if you go to opening eyes website you look at the API documentation and it's like okay how to access our model through the API if you're running a python script well it looks it it's this but instead of like here where it says local model if you're doing open AI it would say gp4 or G GPT 3. 5 turbo or whatever else model is local model because it's running on your computer pretty cool bunkers so I have Python scripts that I could go in and either create like a um i d i duplicate it of course but all I have to do is spin up this server and change this one little thing in here and it would run on mistal 7B mistal or a Lama 2 13B locally and I might do that this weekend you know so uh as you can tell this stuff gets me excited um but yeah you can do you can run your own local LM and I guess you can access it from any I assume it's all on your landan there's probably ways to make it um open to the internet there are services that can do that for you but that's a bit more complicated so yes this is definitely something to mess around with over the weekend or a day or two when you have some time because it's worth learning how to do this kind of stuff if nothing else just to stay current with the technology and of course you have your own local llm here so for example like when I leave here I'm going to go up to the coffee shop and do some work let's say I'm working in the coffee shop and the internet goes out but I need to do some work in an llm well all I gota do is fire up llm studio and I have my own running llm in the middle of nowhere or without internet Okay so it is really really really cool to learn how to do and LM Studio makes it a snap makes it easy to do all right um so uh thanks for sharing valuable information absolutely you are very welcome you know I like stuff like this for everybody because it's so simple that even individuals who might be a little you know shy or worried about getting into this kind of stuff this makes it so easy that pretty much anyone can do it and the worst thing you're going to end up do you could possibly do is download a model that's too much for your system and it'll warn you several times like hey this is too much for your computer right so if I go in here I'll tell you if I do um llama is there a 70b of that yeah so if I go in here and type um Lama 2 70 billions the 70 billion parameter model even on my 4090 it's going to say partial GPU offload possible because this is 29 well it's basically 30 gigs the model is 30 gigs and even my 4090 only has 24 gigs of vram but it'll warn you so the worst that can happen is that you can download a model try to use it and then it'll go the answer to your query is this it goes that slow it'll run but it'll run that slow so it's really good at um warning you if you're over if you're getting out over your skis in terms of size you know but what you can do is when you look for a model you can um download the one that best fits your best fits the criteria and honestly the smaller models are becoming so good like this F 2 is fantastic I think Jim has a little bit of way to go there's a little kind of wonkiness in there but f 2 is fantastic the Llama 2 models are um excellent so I have um I have llama 2 7B here and they're only going to get better someone just came up with a one bit model like

### Segment 10 (45:00 - 50:00) [45:00]

yesterday day before which is very interesting um so try it out just try it out try out F2 try out um Gemma those are smaller ones that can get the job done and basically anything that's three billion parameters or under unless you know you have the the computer to do itless unless you know you have you know eight 12 16 or in my case 24 gigs of vram you know so yeah that's a big model it absolutely is a big model I could probably run that by offloading some of it to my CPU but it would go extremely would take a long time to download so I'm not going to do it now but it would run Mo mostly on my GPU and partially on my CPU and it would just run slow it'll run but it will run slow okay so that's one of the main things I wanted to show you today to try to get I really want to encourage you all to like play around with this stuff and I want to show you some tools that make it easy to do where you can't get yourself in too much trouble um and just start kind of seeing how they work and testing different things and then obviously you will have your own local model on your computer that you can use at anytime anywhere don't need the Internet it's completely private completely secure boom right in your computer all right okay what else we got yes good information you are very welcome you're very welcome alrighty so I think we are um about done I do want to oh I want wanted to remind everybody about this new cool tool to try out I mentioned this in the last um live stream so Google released their notebook LM applications openly now it says it's an experiment but I think this one probably has legs and basically this is a uh a tool that integrates AI into your note taking and so forth so if I just say you know it's an example document here so what you can do is load different Source documents okay like here and it'll summarize them for you like automatically it will it can create different study guides for you challenges it's bunkers okay so it's called notebook llm it's made by Google I would encourage you to try that out especially many of you in the educational space but it's a uh it's a virtual research assistant all right so back in the old days like when I was in school we would go to the card C catalog and find books and then make note cards on them and all other stuff that's kind of like what this sort of does all right but you can load up to 20 sources I think um here and then it will so if I Said So based off the invention of light bulb down here if I say analyze the historical rivalry between so and so I hit that it's going to use Ai and my sources over here and probably its own knowledge to come up with this answer okay and this came from my sources over here crazy it is so cool um and you can um you can add this you to your notes it'll tell you the citations and it'll take you right to where the citations are in your Source documents awesome so yeah it is really cool okay give that a whirl if you have an interest it's just another way that AI is being infused with Educational Tools and content that can be extremely helpful to people all right um of course there look there's benefits to doing it yourself right but I think we sort of have to have a happy medium as they say okay so any last comments questions or whatever before we wind up this live stream I'll be happy to answer those uh before we go I will wait maybe another minute or so and then if no one has anything else we will wrap this one up see here so while I'm waiting for any other comments so as of right now the next live stream will be in two weeks which is March 16th so I might have something to show you then as well we can walk through a different

### Segment 11 (50:00 - 52:00) [50:00]

example or I can show you some of the stuff I've been working on that kind of integrate these tools okay um I think it's really cool to see what you can do and look I'm I I've always said I am not a coder I've tried to learn coding many many many times in my life and once I get a little bit into it I start going to sleep or I run my code and it fails because I missed a semicolon it drive d drove me nuts but now with AI and his AI tools I become this hybrid person where I am a business person like a traditional corporate type business person that has the superpower of AI en coding now and it is it has just exploded my skill set um made me a much um probably more valuable person going forward it's you know bought me some career maybe you never know nowadays some career longevity because I'm learning these tools I'm happy to show you kind of what I've been working on if you want to see that maybe in a couple weeks all right I I'll have something ready to show you probably anyway that I think is relevant to what you all might want to see all right okay bill says going to give that a go on my next course absolutely notebook LM give it a go give it a try it's free so the stuff that's free give it a try who knows how much longer it'll be free all right everybody well it was a good live stream hope I shared some helpful and useful information with you and got you able to maybe run some uh language models locally on your computer that you can play around with and um yeah we'll just keep up we'll keep it up but otherwise I'll see you in two weeks on March 16th barring anything that comes up and I hope you have a wonderful um and productive rest of your weekend all right take care bye-bye