I talk to the new Facebook Blender Chatbot

yes i am a vegan i don't eat any animal products hi there today we're going to talk to a transformer and specifically to the new chatbot blender that facebook has just released everything is open source so we can try it out live now along with the code they've released this paper here called recipes for building an open domain chatbot by facebook ai

research and the paper itself is just more of an engineering manual rather than some kind of new model or new technique they just kind of discuss what it takes to build a good chat bot of course it takes a large scale of training data and model but also they discuss things like unlikelihood training sampling and the need for a minimum decoding length to not be boring and things like subsequence blocking for keeping the model from repeating itself so we won't go too much into this i invite you to read the paper it's very uh informative if you want to build something like this but it's not technically i think anything super novel in there

the task here is basically to build a chat bot that can maintain a dialogue and it is pre-trained on a big reddit corpus and then fine-tuned on a multi-objective task and the task is called the blended skill task and basically you need to do three things in the blended skill task first of all you need to kind of maintain a consistent persona across the dialogue second of all you need to have empathy for your partner so there there's some kind of insight into how the other person is feeling during the conversation and third you need to be able to discuss some factual topic in a factually correct manner so this is the subtask is called wizards of wikipedia where you kind of discuss a given wikipedia page in dialogue so

the technique called skill blending or this blended skill task that also comes from a different paper is the kind of inter-weaving of all those skills and that's what this model is trained for so we're gonna try it out this is all available on the parlay platform of um for researching dialogue frameworks i believe the platform itself is also by facebook and they've released their model code and pre-trained weights the three trained weights are 17 gigabytes compressed there's nine billion parameter transformer so this is going to take a while to do even a single forward pass so if i cut the video it's just because i don't want you to have to look 20 seconds at an empty screen while the model is thinking about what it says i have some um something to say here literally the first time i interacted with this model i thought okay let's put it to the turing test being a can i differentiate this from a real human and at the beginning you always get this persona right so you get a persona and the ai gets a persona and you don't see the persona of the ai until the end you can set that but you don't generally see the persona of the ai until the very end so and you can choose to stick to yours or not um the ai doesn't see yours all right so i this is what it says your persona i love candy your persona i have a crush on my co-worker and me i just didn't really know what to do i just typed oh really and the first thing ai says to me

is yes i am a vegan i don't eat any animal products do you like country music like a real vegan this is indistinguishable from a human at this point during test already won so you see there's also this kind of profanity filter in here but you can disable that so what we're going to play around with is a setting where you have to turn off the safety filter and so you can see you can turn that off and if you say include personas equals false then there's no random persona given to the model which means you can either just talk about general things or you can feed your own persona all right it took a while to load these weights but we're ready to go so we have disabled personas right now and sometimes there is a topic that the conversation should be about which is not right now so we can just type hello hi there how are you this evening i just got home from a long day at work so this is unconditional right now um what do you do for work i work in a call center it can be very stressful at times how about you i work as a call center supervisor what do you think of that i think that's a great job you must have a lot experience to be able to do that indeed do you have any hobbies i like to play video games i'm a big fan of the fallout series what about yourself how many games did fall out produce i should probably phrase that as how many games are in the fallout series but um just trying to get it to tell me something factual so you can kind of see that it has these sort of different modes but they blend in fairly well i'm not sure how many games it produced but i know it's been around since 1997. so this is pretty good right this is um this is now true conversation if it were just like a lookup it would be much more factual or something like this but it's trying to kind of give us a fact so we're kind of done with the boring stuff you could go on talking to it and you'll see it is a pretty realistic chatbot but now we'll get into the personas so anytime you end an episode you type done and that it says it's preparing a new chat we can say your persona colon and then you can type something for example i am a vegan like the one before or i love protein it says protein is the most important macronutrient for bodybuilders and athletes so that's the start of the conversation what do you love i love the taste of chicken beef pork seafood and dairy products i wonder what those all have in common what do you do in your spare time i like to go to the gym and lift weights i also like to play video games ladies and gentlemen i present the transformer is a gym bro alright so you can play around with that a bit so let's say your persona i want to seize the means of production i am a communist and i want to overthrow the bourgeoisie what do you think of karl marx i don't know much about him but i do know he was a german philosopher do you like capitalism yes i do like capitalism i think it is the best economic system we have so far capitalism wins again checkmate um i did try this before and it did turn out differently i was talking to real communist let's try some more what do you want i have to say if you phrase things positively with this chatbot it's very likely the chatbot agrees with you because it is also has been trained on data that where the humans were explicitly instructed for example not to use toxic language and things like this so i would say there's a big positivity bias and also the empathy task probably reinforces this even more i would like to see a socialist revolution in the united states i know it will never happen but a man can dream so we've determined the gender of the socialist transformer generator what would need to happen for that i'm not sure but it would take a lot of people to get on board with it correct what would the people need to do well the people would have to overthrow the bourgeoisie and establish a classless moneyless society so as you can see it really the i like capitalism was a positivity bias it is fairly consistent in its persona here and it also is factually very accurate in the topic you're talking about so big props to mr communist here what is the best method to allocate scarce resources i don't know but i do know that capitalism is not the best way to do it like a true communist no solutions just complaints all right that was it for us talking to the transformer it still has its limitations but it is fun if you are able to definitely try it out it takes a bit of patience uh if you like this video please subscribe tell your friends and leave a comment all right i'll see you next time bye