Introduction to Transformers.js (LIVE RECORDING SESSION ARCHIVE)

All right. Uh, check. One, two, three. I don't know how wise this is to try a second live stream in one day, but I've only got Mondays once a week, so I don't really have a ton of time. It's already 3 p. m. here on the East Coast. So, but anyway, make sure you let me know if you can hear me. I'm going to cut the music off and get started in a second. And um yeah, but just let me know if my audio is okay. Be right back. All right, here I am. Ah, I am not muted so you can hear me. Hi, it's me. I don't have There used to be a button where I would like fade out the music. I don't have that yet. One of these days I'm gonna um have the time to work on improving my setup, but these days if I'm in here, you're trying to maximize the time that I am broadcasting live and talking to you. First of all, huge shout out and hello to Zenova in the chat. If you are not familiar, Zenova is the uh creator of the library transformers. js, which I'm going to explore today. Now, I will say just before your expectations are too high, limited time, this is just going to be a first step towards what I hope will be a much longer series about looking at many different models and capabilities of Transformers. js JS with p5 js. Um, that's the goal today. And I should talk less now because I should just get into the part where I'm like, oh, this is the part that's going to end up in the edited version of this. So, um, but uh, I probably realistically today will just get through what is transformers. js, how do you load it into p5? Let's do a quick hello world demo of like a first thing you could try with it and then

maybe one other demonstration. The topic in my class is using a language model, so I might jump right to that. Um, okay. Great times for Europe these streams today. Well, that's nice to hear, Vincent. Um, all right. So, without further ado, I think I'm going to try to skip with all the pleasantries that I've been doing in these live streams and taking up so much time. I'm just like looking around to make sure everything's going. My voice is a little scratchy because I streamed for three hours already today. Took an hour break. Here I am back. But, um, and if you missed the stream earlier, it was all about loading data in p5 2. 0. You can go back and watch that. Um, and uh, yeah, Mark is asking a great question. Let me just address this if I can find my mouse. Um, do you post the class homeworks online so we can try them out also? So, uh, yes. Um, so, um, if you want to follow I don't know why this button doesn't work now. I think I might have to go to here, then press this button. Uh, oh. Okay, wait. Great. Okay, I think we're good now. Um, if you want to uh follow my class from this semester, um, it is here programming from A to Z A2ZF25. Um, and so you can look through the syllabus and you can click on any of these topics. For example, this is what I will be going over uh, tomorrow in class. And I really I want to make an example. I really want to do today. I don't think there's going to be time for this and if I start this now then I won't get to transforms. js but I would like to make a video on what is uh the softmax function and the temperature but um yeah okay so please feel free to follow along. All right. So, I'm just gonna get started. Um, and okay, so for anybody who wasn't here this morning or hasn't tuned in recently, these live streams are meant to be watchable live. They aren't they haven't been so watchable to be honest though. And they are meant to be watchable as an archive, but they are very raw and I often have a lot of difficulties and take a lot of time to like look stuff up or like repeat stuff sometimes. And if that's not for you, you can just wait because I then I mean I haven't succeeded at this yet, but the idea is then there will be shorter edited, you know, tutorials that come out of segments during the live stream. So, for example, this morning there were two of those that should be edited and out later this week. Okay. Um I'm just letting people kind of trickle in. Seeing if there's any other questions. I have this whiteboard which is kind of the bane of my existence now. I don't know how well this is working. Oh, do not want you to see my uh code. And just in case I end up using it, I'm going to have it be open. Um just going to get this set up. I probably won't use this. It hasn't been going super well and I don't think it's necessary for today, but let's just make a new canvas just in case. And I'm going to click uh here. And I'm just going to click this ecam live start new recording. So that should be going now. Okay. Hi everyone, welcome to a first video in a new series on the coding train all about working with machine models in JavaScript with the transformers. js library. So buckle up. I don't know why I said buckle up. And also I was not recording the right screen. Not that it really matters because all the pieces are getting recorded. But so everybody, let's say it all together. I will not repeat over and over the same thing. It's fine to have verbal flubs and wander around and just talk and show stuff. It can be edited. And even if it's not, it is what it is. Because you know what? Especially with something like this, transformers. js JS is an incredible library, but I have a feeling in a few months, six months, a year from now, is this video going to be that relevant because the interface, the like stuff's going to have changed, you know? So, just do it and then I can do it again next year. It's all fine. Okay.

Um, and the I I'm I think I got to like not look at the chat as much. I like that I have this new system where I can really see the chat. I can highlight stuff, but it is a little distracting. Okay. Um, just doing this. I don't know why those blips happen. I'll use this view for the opening here. I should trick stand here. That's better for Matio because But it's fine. It's going to be fine. Okay, let's do this again. We're going to get started. Hello everyone. Welcome to a new series on the coding train all about working with machine learning models in JavaScript in the browser using the transformers. js library. So this is just the first video in the series. I'm going to give you a little background about what is transformers. js, where does it come from. I'm going to look at how you load it into a p5 js sketch and kind of do some kind of hello world demo as well as talk you through what the range of possibilities of things you could explore with the library are. Then hopefully depending on when you're watching this in the future there will be many videos that follow that show a whole bunch of these demonstrations of different capabilities and different kinds of models that you could use. Um, now I do happen to also have on my channel this entire playlist. I call it a track on the coding train website. A beginner's guide to machine learning in JavaScript with ML5JS. What's ML5? Is that different than transformer. js? Yes, those are two different libraries. ML5 still exists. It's got a vibrant community. I'm working on it. It is built on top of a JavaScript library called TensorFlowJS. I know confusing. They both could be abbreviated as TFJS. TensorFlowJS is a library for machine learning and JavaScript. It is from Google and it runs on top of the TensorFlow machine learning engine. So a lot of the models that I've explored in that series are you know image classification, object detection. More recently, I looked at uh pose detection, 3D pose detection, hand pose, face mesh, etc., etc. What do I want to say here? Um, however, there are many new capabilities of things. techniques, architectures, capabilities, models that can run in the browser that are not supported by TensorFlow and are instead supported by a different library called transformers. So something that might be happening there is uh so I am in fact working with collaborators at NYU and all around the world on investigating how we might integrate transformers. js behind the scenes into ML5. So that might be something that I explore later on this channel. But for now I'm just going to try to figure out transformers. js in its raw form and see how I can integrate it with p5. So, where does transformers. js come from? Well, you might be familiar with You might be See, I'm pausing and stopping. I can pause and take a breath to have an edit point, but I don't need to keep going back and forth. Hugging face. Is that like the actual hugging face in the chat? Oh, that's very exciting. Um, I just had to make a sound. I mean, I should be more excited that Zenova is in the chat, but hugging face to me is like I don't know. It's like some sort of entity. Zenova is a person that I've met who does incredible work and I hope to meet again in the future. Um, anyway. All right. Boy, I feel all this pressure now. I'm going to try to explain. So, Hugging Face, you maybe you've heard of Huggy Face, maybe you

haven't. I would think probably a good analogy is hugging face is like GitHub for machine learning models. I might be getting that wrong, but it is a website repository. I'm feeling all this pressure again, but they're this is too good. Hugging face is commenting in this chat with the hugging face emoji. And I'm looking over here because it's up there. Okay. Uh I got to press this button. Um, okay. Got to move through this and it's okay. I made notes here. Hugging face is an online community and platform where developers and researchers share. That's what I wrote down. I just want to say it's okay to say is the GitHub of vision. All right. I'm just going to go back. I got to move through this. Okay. Hugging face is an online resource community where researchers, developers share a bunch of different things related to machine learning. You can kind of think of it as just like GitHub is a big collection of repositories of code and other kinds of collections of files and projects. Hugging face is has a very sort of similar um uh HuggyFace is similar but focused around machine learning models. And I would say in three categories there there. So if you're on the hugging face website, you can click on this models tab and you can see oh my goodness, there are zillions of models all I got nervous. I lost my mind here. GitHub npm is another good one. Okay, where's server one more time? It's okay. I can use my notes. I told you I wasn't going to start but it's fine. We're going to everything's fine. Hugging Face is a company and their main platform is this website right here. And it is a place where a massive online community of researchers, developers, coders share models, data sets, and something called a hugging face which I will show in a minute related to machine learning. similar to services like GitHub which is a massive online platform for hosting repositories of code. NPM which is a big online platform for hosting JavaScript packages. Hugging face it is the AI community building the future. You can see that uh written right there on their homepage. So there's a lot of different things you can do. You could browse all the different models. We're going to go look and pick and choose some to use. Um it is quite uh wonderful that you they also have this big collection of data sets. The spaces is one of my favorite parts of hugging face. Let's see if we can look at um so uh in a minute I'll talk more about this but Zenova Joshua is the creator of the transformers. js library that we're using. This is his uh page on hugging face and various spaces. So if I were to click on this one for example what a space is, it is a hosted web page. That make sense? Yeah. Spaces is a platform that allows you to host a live web app um to demonstrate a particular feature of a model. So for example, this is whisper which is a machine learning powered speech recognition model. And if I click record, allow the microphone. Choo choo. Welcome to the coding train. And then click load. Oh, and now I click transcribe. Now if I click transcribe audio, it's going to load the model. Transcribing. And then there's what I said and it's time codes. So my goal for this series is all of these interesting, fascinating, incredible demos that you see here in a hugging face that use transformers. js. Can I make those work maybe in an even simpler, more hello world style way in the p5 web editor? That's what I'm hoping to do. One of the reasons why I think using the

hugging face ecosystem in combination with p5 js makes a lot of sense is that hugging face really Um, okay. Uh, thank you. Okay. Uh, well, I appreciate the comments. I'm just going to put this up on the screen one more time. Okay. Um, and I u All right. So, um, one of the reasons why, let me just go back to here. One of the reasons why I think I'm really excited to look at p5 js and hugging face together is that p5 js is an wonderful beautiful online community where there is a lot of trouble today. I don't need to like get the perfect words here. I'm gonna once I get into the tutorial it goes a lot smoother. I think putting hugging face and p5 together is a really um look at the trending spaces. This is very exciting. Okay, what was I going to say? Um p5 js is an so I think hugging face its ecosystem works well as a platform with p5 js. They're both open um evol that you see hosted. Yeah. I mean, I'm trying to think of like what to say about open source, but I think it's fine. I'm just going to I'm going to move on from that. Um you know, what I'm trying to say is that Hugging Face's philosophy and approach with in open source I think is aligned with P5GS and the Processing Foundation's values. But that's sort of a personal take and I don't want to put any words in anybody's mouths. I do not I am not I do not represent hugging face in any official capacity obviously and I actually don't represent p5 js in any official capacity other than I you know was part of the founding of the processing foundation but at this moment in time I'm just a friendly bystander trying to make tutorials and projects and uh spread the gospel if you will of creative coding p5 js and the question that I'm looking at here is what kinds of I think the you The primary reason for this series is to look at you are presumably watching this somebody who is interested in creative coding and p5 js. What does AI offer you in terms of capabilities to help you explore your passions? Um, explore your creativity to express yourself and to the extent that you want to work with open source and share your work in the way that you do with p5 I think a starting point the hugging face repository of models and everything it has to offer I think fits nicely. Oh, without ah okay. So, where does transformers. js come from? Because this is the thing people say to me all the time. Why aren't you do you're doing machine learning in your teaching and your examples and your projects? Why aren't you doing in Python? Well, that's a great question. And the answer isn't, oh, I'm not doing it in Python because I don't like Python or I don't think you should do it in Python. If you're a machine learning researcher, if you are somebody who develops new models, who is working in the low-level uh spaces of creating mach novel machine learning research, then you should be working in Python. And in fact, um one of the things that HuggingFace is most known for is their open-source library called Transformers. And this actually is quite similar. I've talked about this before on the channel. I have TensorFlowJS that I've worked with. TensorFlowJS grew out of the TensorFlow library which is in Python. But so this is something you should learn. Not you should learn this is something you could learn. You might want to learn. But for me what I'm doing is I'm looking at the applications of this research. And I'm looking in particular at what are the applications of this research that you can do locally on your machine in the browser because um yeah always looking for the right wording says Lexarm Onyx. I said that this is like an important detail here Onyx but it's a little beyond the scope of what I'm doing. Um, one of the things that I am in particularly interested in is what does it mean to work with machine learning models locally on your computer with your own consumer hardware.

There are, as you know, many, many, many examples of ways to engage with AI models in the cloud. And many of those systems are closed systems. They're proprietary. You don't know what data they were trained on. You don't if you are in if you are chatting with one of them, your data is leaving your computer and going off somewhere else, being stored somewhere else. And also they're using tremendous amounts of compute. What is the medium? What is the material of these smaller quirky interesting creative models that work with text and images and sound that you can use locally on your computer? And I think HuggingFace is a platform that has many examples of that. And the TransformersJS library I I lost like where I was going. I'm just like rambling here for like no reason. But Transformers. js is a great platform to explore that. So all that is to say, I mean, I went roundabout there. Let me start this over and explain what I'm trying to say in a more coherent way. And then I'm gonna get into the tur. I swear. Why aren't you doing this in Python? The people ask me. Well, I would really like to do this also in Python. And in fact, if you are someone who is interested in becoming a machine learning researcher or engineer, Python is a language you're going to want to learn. And you're probably explore the hugging face transformers library, which is their framework for machine learning models, text, vision, audio, multimodal models, inference and training, all in Python. For me, what I'm looking to do is explore what from all of this research, from everything that all of these people who work on this are doing, what can I pull from there and make work with p5. js in the browser? And what's so exciting about that is when I say make it work in p5 js in the browser, I mean the machine learning model is going to be running in the browser also. No cloud server, no API key, no credit card that I have to pay to like send my data somewhere else. So for me, I'm really trying to prioritize models that run locally on your computer and even those that can run in the browser. Of course, the capabilities are going to be limited, but I think we're going to see you in as I get into these tutorials what kinds of exciting possibilities there are. So, there's a Python library. Maybe I'll come back and explore that a little bit. Um, and the story goes, as I know it, uh, Zenova, who you can find online on GitHub and HuggingFace and elsewhere, uh, was interested, and you know, Zenova is in the live chat right now of me recording this video, so we'll get any corrections. uh was interested in this library and wanted to see if um he could make a version of it that ran in JavaScript and he started working on it and made a version that runs in JavaScript and it got really popular on GitHub and so HuggingFace said that's really great we should have a version of this that runs in JavaScript and now Zenova works for HuggingFace and the Transformers. js JS library um is something that he develops I assume with lots of collaborators and other folks uh through the support and um collaboration of hugging face itself. Let's see if I can find transformers. js here. Uh here we go. There we go. Version 3. 73. That's the latest version. Okay. So, that's my little speech on what is hugging face, what is transformers. js. You got questions about that, leave them in the comments. Would love to um answer them or come back and chat more. I'm looking at the chat. Uh okay. All right, so thanks for listening to my little spiel on what is hugging face and transformers. js. Let's dive in and see how we can make this work in p5. Okay, I'm not signed into hugging face right now, but I think that's okay. Okay, there we go. Perfect. Let me come back to here. I want to see. Yeah

I want to talk about the pipeline, but that's fine. I think I'll talk about it in the context of transformers. js. I should probably log in. Let me just log in. I just have some like private models and things on Hugging Face that I don't want to necessarily show up, but it seems like it's like constantly showing asking me to sign up. So, I think I should log in. Um, so just give me a second because I'm going to do that. Um, uh, and then I'll get into the more tutorial part of this. okay. Let's see. Okay, wait. I have to go to I'm like lost what I'm doing. Um, just give me a second. I swear this won't take too long. We have like an insane password as one should. Okay. I am now logged in to hugging face and I'm going to go to transformers. js and there we go. Great. Okay. All right. Let me put this back. All right. So, you want to get started with transformers. js. This is the documentation page. And of course, even though I'm just rambling here, one of the things I will do, and you're watching this back later. All right, so you want to get started with transformers. js. Here's the documentation page. Of course, uh, take a look at the video's description. I will include links to all the different things I'm showing you here. There's a lot of great reading for you to do. What I want to focus on is this. So this is what makes transformers. js library special. This particular API called the pipeline API. And by API I mean what do I mean exactly? Well API stands for application programming interface. But in this case I mean the framework the sort of foundation of which every thing that you might do with the library works. You always first have to create this pipeline. So this is Python code but it you can see what happens. You are importing the library, you're creating the pipeline and then you're sending some data into the pipeline and getting a result. And this kind of hello world version is for sentiment analysis. And maybe that's what I'll do in JavaScript. And of course, if I scroll down here, we can see um if I shrink I see. I only could see this. Now, if I scroll over here, there we go. now, there this is the JavaScript version. And you can see I guess I'm going to shrink the screen here a little bit so I can um zoom into this. Now you can see ah import pipeline. Great. We're importing the library await pipeline. Okay, we're allocating a pipeline for the task sentiment an analysis and then we are passing some data in and getting a result out. Here's the thing. If you are a p5 js programmer, there's going to be some things here you don't recognize. Import. That's not syntax you've ever seen before. What are those curly brackets doing there? What's this await keyword? Const. Do I need const? This is the stuff I'm going to talk about and cover in this video. So, let's actually just take this and let's bring this into a p5 js sketch and let's just put it in setup. Right now, the dream here is that I could somehow just get this code to run right here in the p5 web editor. What do I want to say? So, first let me talk about something. If you are watching this video, you will want to update your version of P5 to 2. 0. I just happened to make I just happen to have made a whole long video this morning, which should also be out on the channel by the time you're watching this, all about updating to P5

2. 0 and how P5 2. 0 know supports this keyword await which is something you need to work with transformers. js. So um the too long didn't read version of this is just go click this little chip in the top right of the p5 web editor and make sure you've selected a 2. 0 version. I've already done that. And if you want to learn more, go watch that other video that I'll link in the description. Now, if Okay. So, but let's before we tackle the await issue, we've got to tackle this import. I have some notes. Yeah, I made notes. No server needed, privacy focused, interactive. Those are the things I wanted to say about JavaScript. Um, okay. So, JavaScript, JavaScript. Oh, JavaScript. As you might know, or if you've watched me for the however many 10 plus years I've been doing this, there's always 1,500 different ways to do the same thing in JavaScript. And JavaScript as a programming language is always changing and updating and having new features. It's not really that new anymore, but it might be new to you if all of you have used p5. The way that you load a library in the p5 web editor is by looking at your index. html file and finding the script tag. It's loading the p5 library version two. That's called global mode. Maybe I think that's global mode. whatever. There is something else called ES6. ES6 being the version of JavaScript that we're talking about modules which support something called import and export. And those are ways to bring in another JavaScript file from in your current JavaScript file as well as export something you're coding in your JavaScript file to another JavaScript file. That's what you would use to make libraries and engage in a like broader JavaScript ecosystem. Transformers. js uses import. P5JS does not use import. So they aren't a perfect puzzle. match in that sense. No problem. There is a way we can get around this. We can ultimately keep the global mode script tag and rewrite this import statement in a different way to essentially call an import function to bring in the library. And so instead of the import keyword being there, I'm going to say let then I'm going to declare this variable pipeline and I'm going to set it equal to await call a function import which is supported in JavaScript. And let me close this out so I have more room. And I'm going to I'm gonna talk about async and await. This should work. It's essentially saying uh actually it's not going to work. We're missing a detail. But hold on, I'm going to get to all the points I need to get. We need to have access to the pipeline in our p5 js sketch. The place we get the pipeline from is from the huggingface transformers library. So we're going to import it and we're going to use await and async to do that. However, this at huggingface transformers is not specific enough here. Let me just run this. I think this won't work. Yeah, it can't resolve this. Right. Ah, one of the things about modules in JavaScript is they've got to look up from some repository. I don't know what I'm saying here. Let me just make this simpler for the purpose of this video. So it's not four hours long. This video will be cut down, but I don't know about to a concise video to answer Wazardium's question. Okay, because I'm doing it this way, I can't just reference the library through at huggingface transformers. I need to use the proper URL just like the URL I'm using here for the P5 library. And I have that in my notes here. I'm just going to type it out, which is a little bit silly.

How am I doing here? hugging at hugging face slashtransformers and I'm going to do something here at 3. 7. 3. I think it'll work anyway. Okay, great. Bring this back here. There we go. Now I am bringing in the library from its from the JS Deliver web server and I'm bringing in the transformers library from hugging face. I am tagging a very specific version at 3. 73. You don't have to. It will automatically pull the latest version if you don't do this, but I just want to be really explicit in the code and have people see which version I'm using at the time being. Let's make sure this works. All right, we are going to come over here to the whiteboard. I actually do need the whiteboard. Um, and we're going to call this I know you can't see me right now. Don't worry. I'm going to bring you over here with me in a second. Um, okay. Okay, because I need to talk about Let's bring this. Um, and let me also do this. Nope. Uh, ecam. Uh, start new recording. There we go. Okay. All right. So we're over here. All right. So I'm going to explain await and async in just a moment. But before I get there, I want to talk about this syntax here. Pipeline curly bracket curly bracket, which might be unfamiliar to you. And this is as good a time as any for me to explain it. Okay, I got to press the button. Let's say I have an object in JavaScript that says inventory curly bracket close curly bracket. What is an object in JavaScript? It is a collection of key value pairs. So maybe I have a property called mango with a value of five. A property called blueberry. Oops, I hit the wrong button. With a value of 100. One more. Let's say apple. And I have nine. So this is JavaScript object syntax. I have a variable called inventory. It equals an up. I don't need to say this. This can be cut out. This is JavaScript object syntax. Okay. Let's say I wanted to get the value five into its own variable. I might say this. Let mango equal inventory. mango. This is saying look up. up the object inventory and then go to its property mango and give me the value there and set it equal to this new variable called mango. It's essentially pulling out this individual property and putting it into its own variable of the same name. Well, this is a common enough thing that you want to do in JavaScript that I can actually do this. Let curly bracket mango close curly bracket equal inventory. This is known as object dstructuring. I'm essentially taking a property called mango out of the object and putting it in its own variable. And in fact, one of the lovely things you can do with this is you can do it with a bunch of things.

Now I have basically taken. Now I have essentially taken this object and deconstructed it into three different individual variables. This is exactly what's going on in that import statement. In other words, importing that library is a giant collection of things. Pipeline, I mean, the only other one I know is text streamer, but there's all these properties. All of the functions and classes and things that the library can do, they're all sitting there in this giant object. The only thing I want out of it is the pipeline. So I am just taking the pipeline property out of that library and putting it in its own variable. That's what's going on there. Okay. So that's what I wanted to say about that. Now let's have a little quick talk about await and async. Again, I just I have a very long video which I go through and explain that in a lot of detail. So, you're welcome to go and watch that. But what's going on here is in P5 2. 0, the way that you can load things in setup um is using this. I'm trying to think of what I'm trying to say here. Um oh, and I'm getting a like super chat here, which is very unnecessary, but I will put it up on the screen. Udep, thank you very much. That's very kind of you and very much appreciated. Um, okay. What am I going to say here? What time is it also? 3:50. I'm doing all right. Okay. I Oh, hold on. I'm going to pull up one of my sketches. I know what I'm going to do. This will make sense. Just give me a second here. Loading data. Uh, loading image async and await. Let's try this. Yeah. Oh, no. I'm going to have to recreate a lot of these examples because I was like in the video I was recording earlier, I didn't necessarily uh Okay, so hold on. I'm making a quick example here. Uh, okay. I don't I probably said all I knew. Okay. Let's have a little talk about await and async while we're here. I made a whole video earlier today that I will link to where I go through and look at different kinds of things you can load in p5 and how you do it with await and async. The quick version that I'll mention now is in p51 there was this thing called preload. And preload existed so that you could load an image and be sure that it was ready by the time you got to setup. You don't need to do that anymore in p52 because p52 supports this JavaScript concept called a promise. So load image the function load image in p5 doesn't actually load the image. Well, it does load the image. It doesn't actually give you back the image. It gives you back a promise. So there's the image itself and then there's the promise which is like the idea, the promise that this image will be delivered in the future. So you could get just the promise, but that's not what you want. What you want is to wait for the promise to be fulfilled. So by calling load image, getting the promise, waiting for the promise to be fulfilled, the data for the image goes into the image variable. And with that await keyword, you're then guaranteed that the image will be ready by the time you get into draw. And because promises are a JavaScript way of handling asynchronous events, if you're going to handle your asynchronous event with the await keyword, which you want to do because it's very convenient, then you just need to label the setup function as async. So that's something new that you'll want to

add for any examples that load data asynchronously or if you're using another JavaScript library like transformers. js that needs to load the library asynchronously in setup. Got it? It's a good thing I recorded that whole video earlier today because I was able to explain that like really fast. Okay. All right. So, let's go through the rest of this and see what happens now. Oh. Um so we didn't ah okay I'm realizing so we got the pipeline from we got the idea of the pipeline from the library now we need to create a pipeline so there are lots of different kinds of so when you create a pipeline you need to specify certain things. The only thing you are technically required to specify, although it's a good idea, as you're going to see as we do more examples, to specify a lot more, is the task you want to do. So, I want to create a pipeline for the task sentiment analysis. And the reason why I don't love this is one of the things that's important to me is to demystify this technology. It's not magic. There isn't some like magic canonical universal way to do sentiment analysis. It only exists because there's a very specific machine learning model that was trained with specific data that is doing sentiment analysis in a certain way based on its model architecture and how it was trained. So I really want to get very quickly to the point where we need to specify the task and what model we're using. But transformers. js will pick a default model behind the scenes for any given task. So in this case we allocate we create that pipeline. We have to do this asynchronously with 08 for the task sentiment analysis. And const is just a way of creating a variable that you might not have seen if you only use p5. Um that enforces that the variable cannot be redefined. No, that's the wrong word. It cannot be re It cannot be what's the right way to say this? I say it's a constant. It cannot be redefined. You can't redefine a variable anyway. It cannot be changed. But see, there's a lot of like funny business with that because you can change the internals of an object. I don't know the right way to say this. Somebody in the chat's going to help me. I mean, I made a whole video about let and const. So, I don't know why I can't. I guess I should say. I think I'll just say it's re reassigned. Thank you. Reassigned. Const is a label for declaring a variable that require that in ver const is a label for declaring a variable that require that makes it so that yeah it can't be reassigned. Const is a label for declaring a variable just like let the difference is if you use const you can't reassign that variable. So, it helps you sort of protect yourself, protect yourself against yourself. If you know this variable should never change, it's a constant. There's a lot more to it. I made a whole video about it. It's not that important. I'm going to just be my myself. I'm me and I just going to I'm just going to use let we're going to live with let results instead of out. I like to say that. So we create the pipeline and then if I have that pipeline I can now send data into the pipeline and of course there's often a lot more properties that you need to send along with but in this case for sentiment analysis what it's doing is it's taking a block of text and it's giving it a score between zero and one for being positive and zero for being negative. And again it's important to ask the question this is not some universal truth. What the sentiment of given languages means so much to different people in so many different contexts. This is just one specific take on that according to this model and the data it was trained on. Okay. Let's run this and see if we can console log these results. Okay, so a couple things you'll notice. One of the things when you're using transformers. js is you're going to see a lot of different warnings and messages in the console. Don't be alarmed. Sometimes those messages might make you think that everything's broken. They're not necessarily. It's just like, hey, you're using this computer with this GPU, and I kind of thought I prefer this one, but I'm going to do it this way. And you know I like big numbers with lots of decimal places but let's just

use a smaller decimal place now for this reason. Like a lot of messages like that with machine learning data will populate the console. Don't be alarmed. Okay. So first we're seeing no model was specified. It's using the default model. So one thing I would say is let's not use a default model. Let's go right here and put that in next. The second argument to the pipeline is oh and um is the model you're picking. And I'm going to show you how and why you might pick different models. So we can see here we're getting the model that is in ZOVA's um account on hugging face and it is called distilbert base uncased fine-tuned SST2 English. Excellent. Great model choice. So now we have that model. Now it is saying DT type is not specified for the model using the default DT type. Okay, actually now it's giving me another message about DT type. Let me come back to that in a little bit. Let's just look and see if we got the results we wanted. Positive. 99% positive. Awesome. We did it. We have transformers. js working in p5. Let's make this interactive in a quick and dirty way just to have a little demonstration. I'm going to keep the canvas. I think it's sort of fun to have the canvas. I'm going to turn on auto refresh. Close this so I have a little more room. Okay. Am I standing too much in front of it? Move this over a tiny bit. Okay, let's make the canvas smaller. I'm going to quickly just change the background color of the web page. preview so you can see it more easily. And let's make a text input field. This is using the p5 DOM functions. I'm going to say create input. So I've got a input box with unicorns and rainbows as the default text. Let's make a submit button and let's handle that submit button with a event. Now, there's all sorts of fancy ways I could write the JavaScript to keep in line with modern conventions, but I'm going to use a p5 style. And just lower down here, I'm going to write the callback for that function called analyze. And in this function, I need to do what? I need to get the text from the text input box. So, I get that with value. And then I need to get the results from the pipeline, but send in my text. And then we're going to put those results in a paragraph element to display them on the page. But because I went ahead and made this Oh, and we're seeing an issue here, which I'll get to. Because I went ahead and made this function outside of setup. I need access to text input and the pipeline. Those need to be global variables and it was called txt input and pipe. So make that a global variable and make pipe a global variable. But we've still got a problem. Await is only valid in an async function. So setup is not the only function that could have a promise being fulfilled inside of it that you're waiting for. So anytime you write any function anywhere in your code that is going to use the await syntax, you've got to label it as async. So I'm going to add that in here. And now I'm pretty sure um that I can hit submit. And what did I get wrong here? Oh, text input. I called it txt input. Okay, so these are my results. You can't just put a JavaScript object in a DOM element on the web page. It just thinks it's a bracket object object.

Remember the object has a label and a score. Guess what? This is a good time for us to use object dstructuring. I don't need the results. I want the label and the score pulled out of the results as individual variables. I can create a paragraph with the label. Let's just Does that not work? Undefined. Why Hold on. Oh, it's an array. Okay, that did not work. You know why? I wasn't paying close attention. This is a very common thing. So, transformers. js. So the pipeline does not assume that you might only give it one thing to analyze. You might give it like five things to analyze all at once. It's always going to give you back the results as an array based on the number of things you've sent into the model. In this case, I only sent one thing, but you can see the results are in the zero element of this array. So I need to be a bit more careful. And I think let's stick with getting the results. Then I can say label comma score equals dstructure the object that's in the zero element. And now when I hit submit positive and why was I doing this? I thought well just as the for demonstration purposes let's use lurp color. So, I'm going to create a color that is green. And I'm going to say uh P color for positive is green. Uh N color for negative is red. Could be I could choose different colors. And then I'm going to say uh let C equal lurp color. So, lurp for linear interpolation gives me a color in between two values, two colors where zero is all the way one color and one the other color. It's a little bit of overkill here. So, I'm going to lurp between the negative color and the positive color based off of the score. And then I'm going to draw that background in the canvas based on that color. Just wanted to make a little bit more of an interesting demo here. So, we can run this sketch and I can hit submit. Okay. Create color. There's no function called create color. I just made that up because I'm tired. I'm on my like fifth hour of live streaming today. It's just the color function. I don't know what I was thinking. Let's hit submit. There we go. Very positive. Now, let's just sort of see. Oh, I forgot this, by the way, is unnecessary. That's just extra code. Now run this again to make sure that wasn't affecting anything. Great. Let's put something else in here. I'm not sad, but I am tired. Negative. Oh, why didn't I get a new background? Oh, okay. I'm gonna leave this as an exercise to you because it's a little bit more complicated than I thought. I'll show you. I'll show you why. I have an idea. Let's console log this stuff. Okay. Going to run this. Unicorns and rainbow is 99% positive. I just assumed I'm going to get a low number for negative. But no, I got the label negative and the confidence that it's negative. So, I need some sort of if statement or maybe lurp wasn't the best way to do. But you can figure that out. You can make your own much more creative and interesting version of this. That's not what's important here. Okay. Now, what else do I want to say? So let's talk a little bit more about what's going on here when you create the pipeline in terms of choosing the task and the model and this other message

here that says dtype not specified. Let's go back to the transformers. js. hugging face website. So again, I encourage you to read the documentation. It's got a lot of information about uh it's got a getting started, it's got tutorials, it's got the reference, it's got a whole lot of information with different blah and you can scroll through and look at all this stuff. One of the things that I like to do when I'm working with transformers. js JS is I will go to the models tab and I will search for models based on some constraints that are offered here. So for example I can click on libraries and if I go and find here I don't see it there it is transformers. js I can click on that. Now, what I'm seeing over here to the right are only the models that work with transformers. js. There are 2,238 of them. So, you know, that might be hard to just randomly pixeling out of there. Although, small LM2135M instruct is one that I might be interested in exploring in a future video. However, I could also then navigate over to tasks. So you can see here all the different tasks multimodal computer vision natural language processing audio etc. Let's look at sentence analys sent I guess it would be under text classification. What did um what did I use in my code? Sentiment analysis. How come I don't see a sentiment analysis one? That's kind of interesting. Ah, it's Kate Cristian Vikman in the chat. Nice to see you there. Um, I think I think this late afternoon time brings everybody from Europe into the stream. Okay. Um, I don't know what to say about the fact that I don't see sentiment analysis, but that's fine. This recording stuff is but it's fine. I have a solution now. Just have to figure out how to fix this. It's just an alias. Ah, okay. Thank you, Christopher. Christopher says it's just an alias. So, I think that is um my answer for me. But hold on. I have to uh I have to fix something real quick. Uh new start new recording. Okay, there we go. Okay. So, the task I'm actually looking for is text classification. Um, from what I understand, sentiment analysis is an alias to text classification. So, you can kind of use either and it was nice for the demo to say sentiment analysis, but let's click on text classification. And now I can see aha here is that model that I'm using uh dilbert base uncased fine-tune SSD to English. So now I can look at all sorts of other kinds of models. Let's check out this one that says language detection. Let's see if this works. So if I want to use the language detection model, I'm going to copy the name of the model. I'm going to go back to my code. I'm going to duplicate this sketch. So, I save the previous example. I'm also going to simplify the analyze function and just console log the results since we don't know what format it will come back in yet. Let's remove the canvas. Okay. And now let's change the model needs to be in quotes. And I will change the task to text classification. Let's run this. I'm going to click submit on unicorns and rainbows. We get a result. It is 29% confident it is English Latin fatig I probably need an accent there but there's my poor French

88% confident French lens. So this is what I mean. You actually have everything you need for any given task and any given model you want to use. You just need to create the pipeline. Now different models might start to need different things in terms of what you pass in and the results might be formatted differently. So there's some nuance here, but essentially this template should in theory allow you to access any model that's on the hugging face uh that's a hug that's that's hosted on hugging face that is compatible with transformers. js. Let's talk about a couple more things. Oh, hold on. I since I like this suggestion from the chat, I hope this is not safe for work. We got some Portuguese here. SRD. Is that the correct uh abbreviation for Portuguese? I wonder. Okay, I'll let you all try this yourself. Okay, so Okay, so I'm wrapping up the basic introduction to transformers. js itself. So a few more things I want to cover. One is when you're using a particular model, you probably want to instead of just running and using it, do some research about what it is, who trained the model, what data was it trained on, and what are some of the questions and concerns that you might want to think about in terms of how it could go wrong or be misapplied. the page on hugging face for will is where the documentation will live. One of the things that is not too uncommon that we're seeing here is that this is a port of another model. It's converted to be compatible with JavaScript. But to actually look at the information about the model, it looks like I need to navigate to this other URL which is the original model. Onyx, by the way, you might be wondering, ONNX, that might be a word that you sort of see. That is open network neural open neural network exchange. This is a standardized format for storing the weights, the sort of data of a machine learning model that can work across different platforms. So, transformers. js loads models that are formatted with Onyx format. So that's why you'll often see like oh this other model which maybe was trained in PyTorch has then been converted to onyx format to work with transformers. js. Okay, let's click on this and now we can see this is more about the model. This is the information about the architecture of the model, the data set it was trained on. Um you can see the details here, the data set. This is what I'm looking for. Do we see a model that both has the um that is transparent in terms of what the source code and architecture of the model is as well as the data it was trained on. That to me is a really good uh really important and useful information to have when you're making decisions about how to apply this technology in whatever context you're using it in. Okay. You can see information about how it eval how it was evaluated across different languages and so on and so forth. Let me go back to here. All right. Okay. Um Okay. Two more important details when you are loading the model beyond the task and the name of the model. The other is the device you want it to run on. And this has to do with where are the numbers going to be crunched? what computing resources of your computer are going to be used. Now remember these computing resources are going to be mediated by the web browser because this model is running in the web browser itself. And you might be wondering like what wait where are the model files stored like I don't see them here in the p5 web editor. All I'm doing is like giving this um username and name of the model. It's like sort of a path. So what this library does and you have to be connected to the internet. it um not to process your data but to go and retrieve the model and have it loaded into memory. Later we'll probably see loading some very large models that take

some time to load. We I will need to make examples that have like a progress bar or some type of information. This model this language detection sentiment analysis they're very small. They kind of load so fast we don't really need to worry about that. So the model is being loaded from the cloud just like the p5 library is and the transformers. js JS library is but once the model is loaded from the cloud right here on this laptop the numbers are being crunched inside of the web browser and the web browser could make use of the computer's graphics hardware the GPU or its CPU the central processing unit and so if I want to be spec certain models are going to work better depending on what you do but generally speaking um you might on your first try set the device to web GPU Okay. Yeah. So the way I like to do this is well there's so we need a third property task model and then an object that has properties in it like device and I'm going to say web GPU. Let's make sure this still works. Now, here once I've specified web GPU, here's another one of these warnings that looks very scary and like maybe everything broke. But this you don't have to worry about. It's just giving you some information about like I know you said web GPU, but I'm going to do this part of the CPU because I think that's better. And I'll take care of it. Don't worry about it. I should really put the Let's put the label in the canvas so we don't have to keep looking at the console. And how do you do font size? No, that's does that didn't work for this. How do I make these super big? I guess I can't. I'll just It'll get zoomed when this gets edited. I guess I sp Okay. All right. English. Great. So, it's running on web GPU. And then there's one more property that's useful to think about, and it has to do with this DT type, which is short for data type. Okay, let's go back to the whiteboard. Uh, whiteboard and start new recording. Okay, it should be recording. Okay. Oops. Okay, let's draw a little diagram of a neural network. This is the worst diagram ever. Let me just uh

do that again. Let me do it a little bit more thoughtfully. Maybe we have some inputs. Maybe there's a hidden layer. Maybe there's some outputs. So, this is a two-layer hidden and outputs vanilla neural network of which I've covered and talked about in numerous other videos on this channel that I'm sure you could find and I'll link to in a dense connected multi-layered perceptron neural network. Oops. Every node is connected to every other node. Oh, don't erase this. This whiteboard is the bane of my existence. Okay. There's all sorts of stuff that I could talk about training the model and the there's so much I could go back and talk about again in terms of how the data flows through the neural network and activation functions and all that stuff and training and blah blah. The point of what I'm trying to talk about here is when the model is finished being trained, whatever its architecture might be, is there's lots of different nodes that are connected with a weight. So in this case, there are three inputs connected to four hidden nodes. There are 12 connections or 12 weights. Here we've got four hidden nodes connected to two outputs. So we've got eight weights. So you know there's bi there's the bias and all sorts of stuff that I'm not counting. But you could kind of say like oh this neural network has 20 parameters. You might you know see all these like overly hyped models in the news that have 400 bazillion trillion parameters. That's what they're talking about. How many weights how many numbers are being stored in what is essentially a giant spreadsheet. The thing that you often don't think about is well what is the precision level of those numbers? Are they just like 2. 135 or if one of those numbers is it 2. 135 67 89 1 2 3 4 6 76 81 how precise do they need to be? Are they floating point numbers with 32 bits of detail of decimal places? Are they floatingoint numbers with only six wh ah I keep pressing the erase button floatingoint numbers with only uh I have to press it again floatingoint numbers with only 16 bits there's even a way to store the weight in an integer form and the process of reducing the amount of memory you need to store for each weight value is known as quantizing it, quantization. I don't know. I'll put something on the screen that is the correct term. And of course, in an ideal world, we would all love to have the most precise numbers. But it turns out if you kind of lop off a bunch of decimal places, the results are kind of just as good and it runs faster. So this is that question that you're can make a decision of when you are loading your model. You can choose that level of precision. Floating point 32, floating point 16, Q8 and Q4. Uh there's even a whole guide here where we could learn more about that. Right? Are we using 8 bit or we using 4bit? So in this case it used the default type

FP32 but in a lot of cases you're going to see that in my examples it's going to specify a different data type. Now not all data types will be available for every model. Let's see if this still works. It did. So I qu I'm using the quantize down to fourbit integer data type for this particular model. Are we noticing any difference in behavior or how fast it's running? No. But this is something that really can make a difference. So I'll put a comment in here. So, so that's the full story that what transformers. js JS is how you load the library into p5 using p5 2. 0 how you allocate a pipeline with a task a particular model thinking about specifying what device to do the computation and what precision level of the data and then it's up to you to build your interactive system that uses that model in some way to receive data produce a result and use that result. So, I don't know. Let's see. Maybe from this video, you'll just go off and running and pick some interesting models to try and create some projects. I don't know how long it will take me to build out this playlist, but hopefully whenever you're watching this, you will see some videos ahead of you that show um particular kinds of tasks. I could just name a few that I might try to do, and I can put maybe some images up on the screen to give you like a preview. I might try depth estimation is a good one to try. Uh speechtoext Um uh oh I want to do a lot of stuff with embeddings. Um there's so much you can do with embeddings models and actually uh working with um small large language models I think is what you might actually see in the next video that follows pretty immediately after this one. All right. So thanks uh thanks for watching and I'll see you in the next time. Goodbye. Okay. Um, all right. Okay. So, there we go. I don't really have the time for this, but like Sarah is asking, "Will you have a Q& A after the stream? " I'm a little bit pressed for time right now. I'm thinking about if I have the time to demonstrate um the language model or I might like to get some feedback on this and then come back next week. Thank you Zenova. Zenova's in the chat here. Um hey, it's not over yet necessarily. Everybody's saying goodbye. I could I I'm tempted. Let's just see. Let's give me I'm going to So, one thing I could use help with from people, what's a quick way for me to make this much bigger with CSS? And I know I could ask to tell me the answer, but um I'd like the chat to be able to tell me. Um, I'm not leaving. Everyone's saying goodbye. Uh, every Monday is when I will be streaming. I mean, I probably should stop now. I just feel like, can I in just 10 minutes quickly show the language model? Yeah, the clip is done, not the stream. Yes. Ola Sodland if I pronounced your name correctly. P font size. No, but that would do the paragraph. I don't I did that already. Okay. Like the whole everything should be like if I do this, right? Then that is huge. But why didn't this stuff get bigger? This stuff didn't get bigger like this. Ah, there we go.

Okay, great. And then Okay. I know I No, I don't need more room for it. I just want um But that's not a bad idea. How come text area didn't work? Oh, it's I said that goes first. There we go. Look at this incredible interface design. Okay. I don't know how many of these I think. Okay. So, close this. Close this. Keep that. Okay, let's I just want to see now Um, wait. I want the documentation and I want to go to text generation. Okay, this is what I want to look at. Oh, there's just so much to talk about here in this but because I want to do them. All right, but that's fine. Um, small LM2. Why not do three? Okay. I don't know if I should attempt this now. Um, but because but let's the I'm going to turn into a pumpkin at 5:00 PM, which is 25 minutes from now. So, this might just be practice for me, like coming back next week and doing a more proper tutorial, but let's see what happens. All right. Okay. Uh, welcome. This is another video of looking at working with transformers. js in with who knows p5 js, but certainly in with the p5 js web editor. So what I would like to tackle today is looking at text generation with a language model. And the language model that I am going to use is called small models. A family of efficient and lightweight models from hugging face themselves. Hugging face itself I guess company. Where am I looking for? uh the reason why I am interested in using this model, it is a fully open model. Meaning not just the fact that the model weights, the model file is something you can download and run locally on your computer which does exist for other kinds of language models like the llama models from meta and certainly the GPT OSS models and other examples. What I really love about this model is that the

full training details including the data that was used to train the model as well as its training configuration, all of that is published and available. It supports multiple language. It supports multiple languages and a fairly long context window. All of those things that might be things that I should explain and talk about, but I don't know. I don't have like I was like I could just explain it now. I don't need to talk about the context window. Yeah. Um, we could dive into at some point I might like to look more deeply at what does it mean for it to have a reasoning mode? Why is it an instruct model versus like a regular text completion model? What does the context window mean? But in this video, I just want to look at the sort of bare bones getting it up and running with the idea to see what kinds of creative possibilities um could you, the p5 js creative coder, uh accomplish by having access to this technology inside of your p5 js sketch. Okay. So, if you um in the previous video of this series, I went through all of the fundamentals about how to import the transformers. js library using the import statement inside of a p5. js sketch, which doesn't really support that. I kind of cludged it together and explained how I did that. You can see that right here as well as what is the pipeline API the pipeline framework which requires you specifying a task a model and then optionally the device that the computation is running on whether it's web GPU or CPU etc and the data type the precision level of the model itself which is all stuff that I talked about in the previous video. So in this case what I want to do is look for so now so this particular example just takes something that I type and um let's do this example sends it to what is bam M. There we go. I wanted to get French. This example loads a language detection model, takes some text from a text area, uh, and then gets the results and puts it out. So I could type Sava question mark and it detected it as uh French. So what do I want to do? These are a bunch of the pipelines available with transformers. js and I want to use text generation. So I'm going to change the task to text generation. And as I showed in the previous video, I could go to models. I could select the task. Then I could also select transformers. js. And then now I see a wide variety of models that I could try. There's so much to say about the size of the models. I feel like I maybe I bit off more than I can chew here. Um, I'm going to keep going. So let's start with this small LM2 which is an earlier version of the model. Small LM3. Hold on a sec. Small LM3 is the current version. You could find that and try that. And

One of the things that I talked about in the previous video was around the I talked about the precision level of the parameters of a model. A model being an interconnected network of nodes where every connection there is a weight value that's stored and that is typically some number and at the most precise level would be 32bit floating point. But there's also the question about how many of those connections are there. So when you're using a language model, the more connections in theory potentially the quote unquote smarter that model is. But what does it really mean for it to be smart or not smart? The fewer the connections maybe the capabilities might be less but the size of the model will be smaller and the speed at which it runs will be faster. So that's kind of the trade-off and that's what we're seeing when I look here. I'm seeing like well I could use the 135 million parameter model or the three billion parameter model. There's also this question of base versus instruct which has to do with whether the model itself has been fine-tuned or often this is done with reinforcement learning which is another term that I have not necessarily defined in this series that I'm doing but is the has the model been prepared to act as a chatbot in a call and response kind of style like a conversation or what language models really are and maybe I can make some videos to kind of go back to my you know discussions around marov chains and how they work are really a predicting the next token. So at its base level that base model is just completing a sentence as opposed to expecting a question and then providing a response. So there's a lot of nuance there but that's kind of what you're thinking about as you look between these different models. Let's try this smallest one which I think will be the easiest one for us to get to just run instantly here. So, I'm going to copy the path for this model. I'm going to put it here. Um, I suppose no reason not to just leave web GPU and Q4 right now. Let's just try that. See what happens. And let's look at the results. Now, I might I'm missing an important detail here. So, which I'll come back to. But let's see if I can get some sort of response. Who are you? Well, I got a response. The generated text I got back parided what I said. Who are you? Okay, let's see if we can improve on this. Oh my goodness. So many I think. Oh, let's go to hugging face t. Let's go here. Okay. Yeah. Chat template. I know. Yes. Yes. Okay. Let's do that. Let me I'm gonna um go get one of my prepared demos real quick. Yeah. Okay. Where would the documentation for that be? Was it here? Oh my goodness. Um, text generation docs. I mean, it says uh Well, let's just try this real quick.

Oh, we got all the messages back and it just says I because what's the other thing I need is maybe max tokens. There we go. Okay, great. Okay, so that's what I need to do. Okay. System roll content max new tokens. Okay. Let's get back to where we are. Okay. Well, one of the things these models tend to expect, even though you can just give it a text prompt and different models might you might get better results with it, they actually expect a JavaScript object that has a conversation log in it. So I'm actually going to create an object here called messages. And that should be an array. And the array is uh a list of objects each with a role like system and uh content. You are the host of the coding train. See what that gives us. and then so you can this conversation history can have three characters in it essentially. There's a system which is kind of the overall instructions for the entire conversation to follow. Then there is the user which is the text that is in this text area here. And then now we should be able to send that chat log into the pipeline. And what you'll see is we're going to get back another role which is I don't know what it's going to be called the bot. We'll see. Let's see what happens. Run this. Who are you? Submit. Let's look at the results. So the system message, the user message, and then the assistant. It just says I. So profound. What an intelligent model now. Oh, so what's missing? Where was that page? Uh, if I go back to the documentation, you can see there's a lot of custom priorities that I can put in beyond just the text itself. Temperature, I want to make a whole video about temperature, so stay tuned about that. There's other stuff here that I want to go through, but max new tokens, that's what I need now. I need more than just one token back. I got a token is can think of it like a word but that's obviously an oversimplification. It is a chunk of text. So how many chunks of text is it going to predict to give me back? That's the output of the model. Whoops.

Whoops. So let's create an object here with the max new tokens property of 128. Now by the way you may have noticed that the response I got back had three uh was an array of three things. The full conversation history. What I want when I'm displaying the result in a chatbot is I always just want the last thing that was said. If we were to build this out more, that conversation history would grow. So, I can simply say uh the response might be a good is results. Well, I've got to look at it. I could just look at the example here. Oh, it's not given to me. It's uh how do you do um you do like atgative1 I think that's what it is right so I can actually just get the last element of the array in JavaScript I literally learned this yesterday by saying atgative1 so let's console log just that last message and let me put who are you in the text area when the sketch starts so I don't have to keep retyping it. And let me put the code up here. Let's hit submit. Oh, wait a second. Oh, I get a property called generated text. And then I need the last that's an array. element of that. But look at that. It's a seasoned AI assistant. So, I guess I need to say results. generated_ext atgative1. Oh, I lost it. Let's get let's let's look at the full results. I have an array. The zero element of the array has a property called gen generated text. I need the last element of that and I need its content. So that's going to be I can do this results. Let's the response is results index0 dot oh I have autofresh on generated text so it goes away but I can remember this. generated text at negative1. content content and then create P response. Okay, ready submit. Ah, there we go. We've got the response. How many Rs in strawberry? Let's restart the sketch just so Whoops. Well, that's pretty interesting. Let's see if we can use one of the bigger small three models. I don't know why I can't find it. Let me just find it here in my Oh, yeah. Oh, I'm using small too in these. I don't know why. Okay, let's use a bigger one. 1. 7B. Let's try using one of the bigger models. And I'm going to really turn it up. I'm going to go from 135 million parameters to 1. 7 billion parameters. And let me run this. Now here we're stuck waiting quite a bit. So this model is loading behind the scenes and it's a little frustrating because I don't have any information about where it is in this loading process. Good news and I can add something here. Just going to progress call back X. There is a property called progress callback where I can put a callback function that is going to receive an argument with information about the loading progress. So, I'm going to do it sort of the P5 way or actually it's not the P5 way. It's sort of my style where I'm just going to have

a name function called loading progress and somewhere else in my code. I'm going to write that function. I'm going to give it a single argument called progress. And I'm just going to console log that progress. So now we'll be able to see at least in the console the model loading. The main number to look at is this progress number. I am 25% of the way loaded. I'm going to wait for this to finish. While this is going, I'll mention so you could imagine taking this value using some drawing or animation to have a progress bar. All sorts of things you could do there that I'm just skipping in this, you know, raw sort of barebones demonstration. I'm not sure why it aborted. Let's just try floating point which maybe it needs. Oh, okay. I was okay. What does Q4 F-16 mean? Okay. 4bit F-16. So, I forgot that for this model, uh, Q4 is not supported. I believe it's Q4 F-16. Let's see if this works. It's a special incantation that you need in order to get this model to run. And while I'm doing this live, we have Zenova explaining this to me. This means four bit weights with 16 bit activations. So close everybody. This is like um five hours of streaming today, which might be a it's not a record for me, but it's a lot. Okay, it's loaded. We've got it. It's ready. Who are you? Okay, let's How many Rs in Strawberry? Well, incredible. Three Rs in the word strawberry. Okay. So, my computer is really dying. So now you see the basic fundamentals, the bare bones of how you might load a lightweight open-source language model found on hugging face with transformers. js and use it with p5. I happen to have a collection of four examples that go a little bit further than this. just gonna so I will include links to these in this video's description and I would even

like to come back and cover how some of these work in more detail. But the two things there's two things I want to uh mention. one is that we um but one thing that I think would be really important, is fun to play with that I haven't mentioned yet is that this particular uh way of using the pipeline just sends in the chat log with a maximum number of tokens back gets the result all at once. and then displays that on in the preview or on the web page depending on you know whatever context you're programming this in. You'll notice here that's essentially what's in I could show you this LLM hello world example that I'll include, but that's essentially what I just built. This one says hello world streaming. So yes, I'm live streaming right now on YouTube. You might be watching an edited uh version of that live stream, but that's not the kind of streaming I mean. What this means is streaming from the model. So a language model doesn't output a whole bunch of text all at once. It's always predicting the next token. So there's no reason why you couldn't get those tokens one at a time and start to render them one at a time. So for example, let me run this one. Oh, and uh let me just fix something about this. Um actually, let me just do this. Change the theme to light. So, I'm gonna run this one, which just is also loading the same model, the same data type. It's got a little bit more. It's like putting some of the progress stuff in a paragraph element. Um, it's got a little bit more to how it's doing the send message. But this one of the things you'll notice here is that I am importing both the pipeline and a new object from the hugging face library text streamer. So I can attach a streamer to the model and let's watch what happens. Tell me a joke. So that was happened very fast. Why don't scientists trust atoms? Because they make up everything. I think it's going to give me the same joke every time. This is interesting. I don't know. There's probably a property which I could seed the model with a different random number to get a different output or change the temperature or these other properties that I haven't covered yet. But you can see that's happening. It's very fast, but it's actually outputting one token at a time and showing it. So if I said tell me a long story, this might be more obvious, right? We can see and I I've it's maxed out at whatever max tokens I've given it. But now you can see this is what that streaming result looks like which is a nice way of interacting with the model. And you can also do this with a little bit of extra work. You know, you could stream those responses, give it a little CSS and design and it's suggesting that I take a break every hour or so. Oh boy, I gotta stop now. So, this was a little rough, a little messy. There's tons of stuff I didn't explain and just sort of dropped us into using a language model which comes with a lot of significant important concerns and things to think about in terms of all the ways you might be applying it and in what context and for what purpose. But hopefully you're getting a sense of how to work with this and maybe you have some creative ideas of how you could even work with this in p5 um and take the output of the model for drawing and animation and creative coding and I would very curious to see what you want to do. So there's by when this comes out, at least if you're watching this early, none of the videos that come after this will be there yet. So if there's things you want to see me tackle next, explain, talk about, please leave those in the comments or come visit me and the community on the coding trains discord URL on the screen or in the description or whatever. And I'll see you next time on the coding train. Goodbye. Okay, that's really a goodbye. Um, okay, I gotta get out of here. This was a lot today. Um, thanks everybody for being with me for this very long day of streaming and teaching. I hope this was useful. I I hope Matia doesn't like feel totally overwhelmed. We might need to hire an editing assistant cuz I now have recorded uh four videos

probably each like a half an hour to 45 minutes today and I'm way past being a pumpkin. I'll see you all next Monday. I'll be back streaming next Monday. Continuing beginner p5 2. 0 stuff. More transformers. js stuff. Maybe I want to go back to Markov chains. I don't know. Uh goodbye everybody. I don't know what I did with my train whistle this whole time. That should in case you want to just insert this at the end of the videos. Goodbye. See you next time. Thanks for watching this video. I don't know. There's a few different versions of this. That's it. That's all I've got. Okay, I'm gonna press the finish. Sorry, I don't really have a time for a Q& A. Um, see you got see you all. Goodbye.