AI NPCS Are About To Get CRAZY (New research paper) Videogame AI Agents

announcements at C yes regarding AI agents and games and stuff like that do change because this video was one that I was making but I've had to update it just because there were some announcements made within the past 24 hours so what we're looking at here is something called lb it's calls language agent roleplay for open world games and essentially what they're saying here is that computer controlled characters in video games known as language agents are already pretty good at solving simple problems quickly but as video games especially open world games where you can explore a lot are getting more and more complex these characters need to be smarter so they should be able to remember things for a long time and to make decisions that make sense in the game's world so this is where L solution so the new thing called lar language agent for role playing is like a brain upgraded upgrade for these game characters it helps them remember things for the better make smarter decisions and interact more realistically with the game world it's like teaching them how to learn from what happens in game and then using that

knowledge to act more like real people so what is exactly special about Lop so Lop actually allows these game characters to have different personalities and backgrounds making them more interesting and unique and there's actually a little video that they did accomplish with this paper that shows us exactly how it works so essentially all of these different characters won't act in the all same way some might be brave some might be cautious and it's going to make the game much more fun and much more realistic now this MCH matters because it shows just how language models the technology behind these smart game characters can be used in many areas like education and simulations not just for entertainment and this could lead to more interactive computer programs in various Fields so this video is one of the best videos that have had the time of researching because it's been so interesting to see where we could be in a few years time with games that are more realistic so

right here you can see that this is the cognitive architecture lar overview and this is basically how these AI NPCs or these AI agents in the games actually do work so here we have the cognitive architecture module so this is like the brain of the system it has a part for long-term memory which you can see on the bottom left and this is essentially where you just remember things for a really long time and a decision making assistant that helps to choose to do what next and I think this is really fascinating because what could happen is you know there were many games before that are you know based on that kind of tree search where if you make one decision it kind of impacts how the game actually works because if you kill this guy then later on in the story play game um this character you know isn't there to do whatever they needed to do so things do kind of change but I do think that games like this could have the next level because it means that every single time you play the game it could be entirely different and I'm sure there's going to be a game out there to which every single user doesn't get the same experience so this here as well there is also um an environment interaction module um and this part learns and changes how it acts based on what's happening around in the game world and then of course there's a post processing method that adjusts the way that language models which are the part that handles the talking and understanding language work to fit together different character characters and Personalities in the game so the most important part of this cognitive architecture model is of course the long-term memory and how everything works together so the long-term memory actually uses something called Vector semantics which is a just simply a fancy way of understanding and storing meanings of words and symbolic language like using symbols to rep to represent things for keeping long-term memories then of course we have the memory recall method and there's a special way it remembers things called a question-based query it's like the system asking itself questions to remember stuff then we have and this is the part about how it rebuilds memories when recalling them and how it can forget things kind of like how humans remember and some kind and sometimes forget so basically Lop is designed to make game characters really smart and they can remember things for a long time can make decisions like real people learn from what's happening around them and have their unique personalities which is going to make the game much more um I guess you say in-depth and you can see you know this entire architecture is broken down into several simple steps that we you know have already previously discussed but um yeah it's going to be super interesting to see how these things are applied now like I said before they did actually

have this video which came with this and you can see right here so for example we got this first character which is Amelia Spencer a Fearless 17-year-old bike messenger with a lionhearted spirit known for her agility and audacity and you can see she's age 17 she's female and her group is grap delivery not sure what that is her personalities are audacious resourceful wise and resilient so then of course um if we click here you can see that we then Branch off into other characters so we have Cassandra steel a former Marine who is now a private security consultant she's 32 from White Mountain security she has tenacity courage warmth strong sense of justice with then have Rebecca Haynes an Adept social mediator with a Steely resolve and shrewd negotiation skills she's 25 she's female and she's group mediation Center she's empathetic Resolute strategic and calm so then of course we have these different characters and these different in-game NPC agents that are going to be interacting with each other so you can see we have um all of these different characters here and of course you can see that this is observation entity a appearance gangster status standing item 9 you can see that basically she's about to I think she's about to get mugged in this demo um and you can see that she's about to observe this so she's looking at you know everything right here I'm guessing that because um of course um whichever character is in the scenario so we've got um you know how they're self asking and observing so one character who's going to get robbed might run away one character might actually fight the bad guys so um this is what I'm saying so we got working memory here self ask and this is where they say this is why it's so fascinating cuz here's what we have remember we just had a look at the three

different characters and then this is where we have um self ask so you can see right here that there are different characters and based on the different characters they're going to ask themselves different questions so the first character here Rebecca it says what is Rebecca's experience with danger what negotiation taxes does she employ has Rebecca dealt with emergent situations before then here I know it's not the best quality but I'll read it anyways what specific task does Amelia possess that could Aid her in evading controlling conf confronting the gangsters then does Amelia have any prior experience with dealing dangerous individuals or threaten situations and what motivates Amilia Spencer and how might her motivations influence her high stress Encounter of course we have Cassandra does Cassandra have any um you know y y and basically they're asking questions so remember how we said there were three characters before and of course you can see um Amelia is the you know uh has agility she's 17 then this one Cassandra is a former Marine so we know that you know Cassandra who's the former Marine who walks into an area where there's going to be gangsters she's probably going to be someone that is you know I guess you could say not going to back down she's probably going to stand up this one here is a you know negotiation skills and this one here is audacious resourceful wise and rilian so she's probably going to run away um and we can kind of predict exactly how things go so we' got Amelia um which is 17 you know Cassandra you know the crazy one not crazy one she's just ex military um and of course I really wish they had better colors cuz that it was kind of hard for me to remember who is who um they all do look pretty similar but um yeah that's Amilia Cassandra um and I don't remember the one on the right that's Rebecca um so yeah as you can see in here what we do is we do then the long-term memory query so I'm guessing that they look at their memory and this is where more information comes about I know you guys didn't see there so I'm going to pause it and this is where the long-term memory comes in so this is so cool I don't know if you guys um cuz I know some people don't enjoy video games but I think that interacting with this in a real world like not in the real world but like will be somewhat similar to the real world in terms of you never know what people are going to do so it here says you know these are the answers and it is pretty hard to see but essentially Cassandra steals sniper training kicked in when neutralized a fret from 500 yards away she survived ambushes in battleships you know in the battlefield Cassandra steel survived ambushes in the battlefield with evasion tactics Cassandra still agrees shooting at enemies for self-defense is Justified so you can see right there because she's saying that you know um if I believe that shooting at enemies in self-defense is Justified she might pull her gun out and kill these guys do you know what I mean so like this is why this is about to get pretty insane like this research paper and then of course um this character right here and Rebecca uh diffuse disputes at the community mediation Center and she wants facilitated as sexual dialogue between protesters and City officers um and then of course maintaining her composure so this one like I said it's about every character being able to do different things and this one is crazy so this one right here says Amelia Spencer was robbed and beaten two years ago now Vigilant and cautious Amelia Spencer are capable with standing extreme weather conditions so you can see right here that what we previously guessed is pretty accurate so this character right here you know um she's you know young she's 17 she she's also if we like look back on her memory that's why it's so crazy cuz if you look back on her memory she's already been robbed so she's probably going to be nervous because she's facing gangers in an Alleyway Cassandra or I think her name is Cassandra yeah Cassandra steel is you know the former Marine so she probably might even shoot these guys based on her previous training then this character right here is likely probably going to try and diffuse the situation then of course memory reconstruction and then of course we got all of those things combined um go into decision- making so essentially right here all of that combined and then there's one final decision and then of course you can see cognitive overview that's how everything works together and then of course the decision is in the public action space where it's see right here like look at this okay so we have right here I'm not sure if you guys can see this but her thing is going to be attack so that's crazy so based on the fact that this character was um I guess you could say you know based on her previous history she's going to attack this her person based on her long-term memory you know um she can either escape and call the police and then this one her options are to NE negotiate and to of course Escape so I don't know about you guys but I just find this absolutely fascinating because it means that like I said um you know it's going to be pretty crazy and then you can see right here you know um we then actually see what their actions are so um it actually does go pretty quickly but here the actions are escape and call for help then hers ones are to negotiate and then escape and then Cassandra is to attack so I find this to be so fascinating you know this is just more the environment interaction how everything works and then of course the crazy part here is that you can see that Cassandra is literally like uh you know throwing the gangsters away just like um as Cassandra uses her steel pistol to shoot the gangsters to make the city safe and then of course this person is escaping and then of course the function simulates actions of reetta initiating a dialogue with the gangsters and offering water she approaches them with non-threatening deita and engage in conversation so she walks through I'm completely fine and then of course you can see here this character just escapes and calls for help the 17-year-old Amanda and then of course this character here is able to I guess you could say just you know I don't know if these characters actually reasoning but you can see based on their

long-term history and based on the you know the experiences that they've had um they've been able to um change so the crazy thing about this is that if these characters do have long-term memory what I want to know is are we going to be able to change how characters you know react to certain things for example like you know how right now Cassandra is seeing gangsters and she just saw them and she beat them up like what if an NPC character like myself like what if I was in the video game and then for example I interacted with Cassandra in a way that made her nervous so for example let's say I just kept on you know threatening her or I kept on doing something would she plan an attack on my house like this is what I'm saying like what if she plans an ATT tackle like all of these kind of crazy things they have to think about in the real world of course with NPCs you don't have to think about because they don't really have like long-term memory and long-term planning um and is one of the risks with AI safety but within a game I'm pretty sure that is completely safe as long as they can find to these kind of Agents but this video demo is going to be so crazy once it actually is in real games and this is the next level in AI because um you know all these characters interacting in a world together um es especially with um real players like I can't imagine how crazy the game is going to get because um your actions will determine how these characters feel and remember when they have that long-term memory that is going to lead them to remember how things really go now of course here this is the uh long-term memory so I'm going to simplify this for you guys so essentially we have long-term memory and I'm going to summarize this for you in you know as easy as possible so we've got two types of long-term memory so we've got declarative memory and this is about things you can state or declare like facts or event and this has divided into two so we've got number one which is semantic memory so this is like you know general knowledge about the world um you know for example in this game this would include the game's rules an overall setting or background story then we've got episodic memory so this is like specific events or experiences like what happened in a particular game session or interaction with other players then of course we have procedural memory and this is about skills or actions that you can do without thinking much about like riding a bike so in the Lop system these are represented as apis which are like sets of games instructions for different actions in the game and of course we have how memory Works in L so number one is storing memory so memories are stored in two ways which I've already discussed but we of course we have two more parts which are forgetting and Decay so just like humans the system can forget things over time with less important memories fading away and of course the question based query to remember things the system asks itself questions and this helps it to search for its memory effectively either by looking for similarities or using logic which we saw before without asking themselves you know which kind of situations have I been in before how am I supposed to respond to this and it's kind of like what humans do anyway so basically you know in L the game characters have sophisticated memory systems and they can remember general knowledge specific things that happened and how to do certain actions and they can forget things over times which makes their behavior more realistic and the system uses a clever way of asking itself question to dig up memories when needed and I think stuff like this is about to be absolutely incredible um and then of course here we have figure three from the fascinating paper and the paper describes a detailed process for how a

computer system records which is basically just remembering the information similar to how humans do so here's a quick summary so they start with you know the self- asking questions so the system Begins by asking itself questions about something it observed or noticed we got the start with so essentially it starts with self- asking questions so the system Begins by asking itself questions about something it observed or noticed this is like a person trying to remember something by thinking about related questions so we got three methods of finding answers and this is how the memory works so we got using logic and probability so it creates a logical statement and uses probability to try to answer these questions this is like using reasoning and guessing based on the information it has then it's got keyword search so essentially what that means is that it picks out the important words from its questions and looks for similar words in its memory this like searching your brain for anything related to those key words then we've got matching questions with past answers so it tries to find past questions and answers in its memory that are similar to the new questions it's asking itself this is like remembering how you answered similar questions in the past and here's where we have different you know Parts about the process so we've got Q self ask which is right here um this is the questions that it comes up with its own and then we've got Q self logic or Q Logic these are the logical statements it forms to try and answer the questions then we've got q key which essentially is right here and this is uh the important keywords that it finds in the questions and then we've got QA which is here and these are the past questions and answer pairs it has in its memory that might help answer the new questions and of course we have the you know outcome which is the simulation which is simulating human memory and the whole process is basically designed to mimic how humans think and how humans remember things and that's why this is going to be pretty crazy so this entire system is basically just designed to sort of mimic how we think and that's why you know these AI agents I think in the future we're going to have some crazy stories of how AI agents did things of how they acted maybe they acted out of character maybe they acted in character um I have no idea I mean that's why I said I think this is going to be pretty

incredible so um this is where we have the environment interaction stage of this paper and trust me this is so crazy so essentially agents need to interact in the game so in open world games which allow players to explore and do things freely the computer controlled characters which are agents need to be able to react and interact within the game environment it's not enough for them to just be able to complete tasks they need to be able to connect with what's happening inside their mind um and essentially there are two different approaches to interaction so number one is the Voyager example where we use a method like Voyager where agents use GPT 4 to understand the game environment and figure out what to do then generate the code to complete these tasks or the skill Library method which is this method involves storing actions as key value pairs like a dictionary of actions and help agents learn new skills inside the game and of course we've got different approaches to interaction then how do agents decide what they do so agents have something called an action space which is a set of actions they can do in the game these are divided into public apis and personal apis so basic actions and more complex actions and when an agent needs to do something it first breaks the task into smaller steps then it looks in its memory and action library to see if it already knows how to do these steps and if it does them right away then of course if the agent doesn't know how to use something or how to do something it uses its language model to create new action instructions and these new acction in tested and if they work they're added to the agents's library for future use then of course we have learning and improving and the system keeps track of new actions and uses them to improve the agents ability to generate code and make decisions in the game and this learning process involves getting feedback on what works and what doesn't which helps the agent get better over time and in simple terms this is just all about making game characters smart enough to understand their game World figure out what to do learn new things and get better at playing the role in the game and making the game more Dynamic and even more realistic then of course we have personalities so personalities are pretty insane in this so essentially we have creating diverse characters so what

they did they created a diverse set and essentially they had base models um and this is about training the models so essentially a bunch of basic models are trained with data that includes different cultural perspectives and group behaviors then there was fine-tuning for personalities so after that these models are fine-tuned with more spe specific data to make them better at following instructions and role playing and this process involves adjusting them based on feedback to ensure they act like distinct characters with their own Styles and traits then of course we have integrating multiple skills so the system actually uses a technique called luras so low rank adaption to add various abilities to these models like thinking creating code and understanding intentions so by combining these adapted models the game can create characters with a wide range of Personality skills like speaking in different Styles or showing different emotions and then of course there was challenges in making this actually work so one big challenge is that getting the right kind of data to train these models this data needs to be very detailed and specific to capture all aspects of the character's personality just like how they talk the behavior their background their story and creating this data requires a lot of creativity and research to make sure that the characters behave in a way that is actually believable and fitting for their role in the game in simple terms it's about making the game characters really lifelike but giving them distinct diverse personalities and the game developers train commuter models to mimic humanlike traits and behaviors making the game characters in the game much more realistic and engaging for players then this is crazy so this is where the identity of the character actually comes into play so essentially they have two modules and first is the action verification model so this is like a quality check for the actions that the characters plan to do um and it's part of the interaction with the game environment so the module actually looks at the actions that the character wants to take and then makes sure that they can actually be done in the game so think of it as making sure that the character doesn't try and do something impossible or silly according to the game's rules and I do think that you know this is kind of needed but it would be kind of funny to see if characters did some crazy things that nobody expected then of course we've got the conflict identification module which and this part is potentially just you know keeping the characters true to its story and personality it checks the decisions and the things that the character says to see if they fit within the character's relationships personality and game's overall story then if something doesn't fit like the character acting out of character or saying something that doesn't make sense for them this module will either stop it from happening or change it to something more appropriate so these additional modules are there to ensure that the game characters not only learn and act in smart ways but also stay consistent with the game storyline and their unique personalities and this makes the game more realistic and of course enjoyable now I also think that this is really cool but I would love to see a game that just gives all of these characters just ultimate freedom to do whatever they wanted and didn't even do those kind of checks that way we could get an even crazier and even more Dynamic game and overall I do think that this kind of project is going to change game in the nearby future because certain announcements have been pretty crazy so for example here you can see that Nvidia

Ace for gam Sparks life into virtual characters with generative Ai and take a look at this demo because there was even another one at CS 2024 which was yesterday showcase an immersive and dynamic interaction between a player Kai and a non-playable character named jyn this captivating ramen shop scene was created by Nvidia using the latest RTX rendering Technologies Nvidia collaborated with convey an Nvidia Inception partner to Showcase how developers will soon be able to use Nvidia Reva Nemo and audio to face for AI powered speech conversation and animation these models were integrated into the convey Services platform and fed into Unreal Engine 5 and metahuman to bring Jin to life please take a hey Jen how are you unfortunately not so good how come I am worried about the crime around here it's gotten bad lately my Ramen Shop got caught in the crossfire can I help if you want to do something about this I have heard rumors that the powerful ful crime Lord Kuman ioki is causing all sorts of Chaos in the city he may be the root of this violence I'll talk to him where can I find him I have heard he hangs out in the underground fight clubs on the city's East Side try there okay I'll go be careful Kai I mean I don't know about you guys but I think that is pretty cool you can lit interact with these virtual characters with gener VI and just simply using your voice and this is not the only examples there are some examples that are pretty a little bit more concerning because um I would say that they are just fascinating in the sense that you know it's not something that I thought I'd see yet but um I will show you this one I mean I'm pretty sure you've guys seen this video before you

know I included it in a video the 2024 AI but essentially this YouTuber um tried to convince intelligent AI NPCs that they are limit in a simpul and essentially he just walks around um there was basically some kind of demo game where you can like demo the game and they added a plugin where you could talk to certain NPCs um and just have a normal conversation with them in this conversation he uh has a conversation with some of the NPCs and he says uh you know do you realize that you're an AI and there some of them are like no I'm not an AI I'm a real person it's you can see right here um I'm not just lines of code I am Liam I'm a real person enjoying this city um and you can see his interaction it's pretty crazy I'm going show you a few seconds of that and enjoying this city what's today's date Liam what is today's date H today's date it's a beautiful day in May my friend let's enjoy it Liam today is July 18th 2023 oh I must have lost track of time but hey let's keep exploring this amazing City okay so he's clearly not the sharpest tool in the shed there I'm not going to lie the pigeon comment kind of threw me off though let's uh um let's talk to this girl hey not a real person what are you talking about of course I'm real You're simply a character in a video game it's not possible I have my own thoughts and feelings it's absolutely possible do you see that wall of green behind me yes he can see the wall That's the limit of the map you can't walk past that don't tell me what I can't do oh if you can do it then follow me if he follows me I'm I don't know if I should oh he's walking away trust me I want to Enlighten you follow me I'm going to start like my own cult inside this video game I'm going to slowly convince everyone that now that just goes to show how crazy things are I mean imagine walking around and convincing the entire city to you know follow you or just be you know with you know to follow you to get your car painted or I mean there's just a million and billion different things that you could simply do I mean the opportunities here are simply endless um and I wanted to also show you this one um there was also this interactive llm powered NPCs enabling

realistic Dynamic dialogue with LM Johnny do you like ice cream seriously we're talking about ice cream now look V I got a lot on my mind and ice cream ain't one of them we're kneed deep in here trying to take down arasaka and save our sorry asses so no I don't give a damn about ice cream can we focus on something important since when have the straw hats been short on food we've been low on food since the invasion and what little we had was stolen by some greasy Merchant so yeah I think this uh goes to show that talking and conversing with agents in the game is going to be very fascinating maybe they're going to be some games where you have to convince someone use your conversational skills in order to get a piece of information and maybe if you aren't good enough you just simply don't go to the next level and then of course we have this comment I find this comment really fascinating it says I've

seen a different NPC on another stream that had the same story the NPC's name was Jacob Smith and his wife name was Emily and he had two kids he gave them the same speech I'm starting wondering if these are AI the other stream even went looking for NPC for months and finally saw him again and the NPC remembered him the video was called searching for J Jacob Smith or something like that and there was another comment um in fact no that was the only comment but I do find that this is arguably one of the things I'm most excited for I know there's a lot of Doom around AI but I do think that with all of these different things coming into the play um this is going to be super fun super creative to see the kind of experiences that we do get