Behind the scenes of building an AI coach – Hackathon team retreat storytime

I recently participated in my first ever hackathon and I want to tell you all about it it's a story with highs I'm using a tool called replit and it is genuinely blowing my mind with lows not as exciting when it runs into failure and also crocodiles so sit down get cozy and let me tell you the story of my first ever hackathon experience at the kit create week Retreat this retreat was taking place in Mexico and my journey there started late on Thursday night it took me three flights around 24 hours but I finally made it alongside Dan our product design director who also happens to live in Spain if you haven't heard me talk about my work before kit is a fully remote company and we get together twice a year in person normally our Retreats are filled with strategy sessions and brainstorms on lots of different topics but we were trying something a little bit different this time and running a companywide hackathon so everyone was participating in it wasn't just for like the engineers we stayed at a

beautiful resort in Puerto Vata my room even had it own jacuzzi on the balcony and I had the whole of Saturday to recover from jet lag relax by the pool make friends with the wildlife and even go to the driving range at the golf course that's attached to the hotel here's the thing I just started learning to play golf in August and I've been to the driving range I guess a couple times since then to practice but I'm playing my first actual round of golf with some team members on Tuesday I've been assured that I'm not going to be hindrance and that these ways we can play that like you know won't require me to be very good essentially but I still I'm like I want to be able to hold my own somewhat and so I'm here practicing also was just fun to practice at our Retreats all the managers fly in a day early CU we have this manager Day on Sunday before the retreat kicks off and anyone who lives in a far away time zone flies in early to have a recovery day like I was doing and at dinner that night there was a magician named spider and his magic tricks genuinely blew my mind it was very fun what anyway the next day we had that manager day over at the beach club just go back from a full day of manager sessions we did a SWAT exercise strengths weaknesses opportunities threats if you haven't heard of it um where we were putting like how we're doing as a company basically taking that high level View and then picked a couple of topics to dive in deep on do this really fun exercise for like generating conversation or like sharing opinions or whatever drawing topics like that where we have these four boards around the room they say strongly agree disagree strongly disagree and like someone will say a statement and we move around the room to where we land on that statement and it just really helps like get a lay of the land for what folks think about our conversation instead of like having to sit around a table and everyone take time to share their opinion you can sort of like get that initial sense really quickly so yeah it's a great exercise but now the rest of the team has gotten here and so I'm going to go down to the happy hour and uh hug a lot of people I guess it's always so nice to see everybody after 6 months apart and on the first night of The Retreat on the Sunday night we all get together in a big circle and we share celebrations of things that happened over the last six months things we're proud of and my favorite part is we open our profit sharing checks which is always a great time on Monday we had a few scene setting talks from our CEO and our VP of product we met in our individual teams for one session this is my wonderful brand Studio team minus one person who sadly couldn't be there we missed you Henry and then that afternoon create week officially began we are

getting deep into our create week projects today yesterday we had a kickoff session that my group used for like vision and goal setting of like where are we going to get to by the end of the week let's try and keep this within scope so we can actually finish something we made a plan for how we're going to spend each of the sessions our project if I haven't said it yet is to build a Creator coach AI so we want to train uh an AI on a bunch of our content about Creator businesses a bunch of Nathan's content from his podcast where he's like coaching other creators on their businesses and we want to see if we can build something where a could use it to like help them launch a product or like create a course or strategize about the next move for their business if they could do that in partnership with our ai why is that so hard to say based on yeah this content and our expertise that we've built up as a business then that would be really useful so yeah we started Gathering the material to like train that Ai and I'm saying this like I know what that involves but I guess I'm going to figure that out today so a bit of background on how these create Ro projects work before the retreat everyone had a chance to pitch projects like what would they want to work on what their ideas were I think 14 of them were selected to be worked on um and one person was named as the leader of each project and then everyone else could pick like which project they wanted to contribute to during the week our project leader was Miguel from the customer experience team and we had a real mix of folks in our group from other teams so it was really fun for me to get a chance to work closely with people that I perhaps don't normally get to it closely with but and this is a very important point for later on we did not have anyone from engineering on our team so keep that in mind as we get into the details of this creative coach AI project we've just ended first session of the day where did we get to do we feel like we're on track we got like a lot of the source material collected almost all of it we have some stuff going and we're like testing out like training the different models and we've got a very ugly prototype website just turn the screen around and you can show the your prototype yeah here we go even says kit logo at the top so we're going to make it Park bying GP to make it and I've got going on I'm using coach boox AI I've fed at some things and now I'm basically in the stage of like seeing what it's given me making progress these right here I'll put them on screen were all the training materials we gathered and on the first day we were testing a few different SAS tools that were designed to help you make an AI coach as well as setting up a chat GPT assistant and we were looking to see which tool felt like it would get us closest to what we wanted with the level of control that we wanted to have um and of course we started running into issues sometimes we'll have failures depending on like the like there's some limitations to how much yeah this happened before so give me a second so here you can you limit the return not as exciting when it runs a failure oh exciting times ahead that doesn't sound like us at all these are not exciting times it's hard cuz this is also a new like I don't know some of these yeah and I don't know what settings I don't know stuff about like tokens and minimums and maximums I don't know what we're spiraling we had to leave it there for the day though because on Tuesday afternoon we had an activity time which is where I had chosen to play that first

round of golf that I talked about earlier David on the brand Studio team is a very Avid golfer so this is him telling us that we're going to play a scramble which I had no idea what they meant at the time but um I soon came to learn it's perfect for a beginner cuz basically you just like hit from wherever the best wall in your group landed so I didn't have to actually hit my walls particularly well although one or two times my ball actually was the best ball so very proud of that how do you think it's going to go David I spectacularly well oh is this your B little did I know then that this iguana would not be the biggest reptile that I saw out on the course I'll spare you the like 30 minutes of golf footage that I took this afternoon but this was so much fun and a big shout out to these guys for being so encouraging during my first game and for making it so fun this is definitely the start of a new hobby for me I loved it so much but this is not a golf channel so let's get back to the work slightly spooky lighting but it is 7:00 a. m. I

had so much fun at golf yesterday that I want to go again so I'm going to go to the driving range this morning I'm just waiting for it not to be so dark outside um and while I'm doing that I am getting a bit of a head start cuz today's our last day to work on it and I am using a tool called replit ret I'm not actually sure how to pronounce it for the first time and it is genuinely blowing my mind let me show you what I have been making what I'm trying to do is like mock up a way we think the AI will live which is in this if you hit command K inside the kit app um we get this little model and so we're thinking this is where the AI could live and so I'm trying to mimic that and this is what I've got so far here in replit I've got it looking like this we've got the logo here which is matching here um I've got it saying chat with your creative coach I just asked it to change the color of the chat bubble so we'll just test that briefly not an actual prompto give ooh interesting okay I told her to make the text black and so it's done that a little too well that's good that it asked me a question I did ask her that to be blue and I asked for this to be our like brand blue which it's not so come back in here I'm going to copy from our brand guidelines how should I say this text in the black bubble should be oop let's see what this gives us so it takes a bit of time but it's pretty cool that I can just you know tell her to do things rather than having to Google how to make that change myself when you see the screenshot you can immediately tell whoops well it didn't respond to that but okay let's test the bubble text at least good okay that's the right color now maybe we make this blue this process of coating quote unquote I don't know if we uh like want to say it without quotes yet but it made me feel so powerful just like typing what I wanted it to do and having rep then like implement it for me it was just like it honestly blew my mind yay now it's blue like I wanted it to be it's so fun to like work with AI on something like this is how we need to be seeing it you know it's not something that is going to replace our jobs it's something we're going to use to do our jobs and it's interesting to see how it does require some sort of like understanding of what you're doing as well like you know how I want that send button to be blue I know that it's light blue because it's disabled because I understand like how software works and when you can't click on a button like the general pattern is for it to be faded so the AI is probably making a good choice there it's just like not the look that I want but I can imagine if I didn't like understand that would be frustrating I'd feel like oh it's not doing what I want it to do if I wanted that to be our full strength blue I would need to say don't disable the send button I'm not going to test that right now because I am going to go meet David and go to the driving range but yes we have three sessions today how many hours will that be probably going to be at least 5 hours that we'll spend on this but we need to finish our mockup there's definitely still some like training of the AI to be done and I want to think about how to like position and Market it and of course I'm going to use Claude my best AI writing friend and we're going to practice our presentation and make the presentation to begin with I guess there's a lot of work to do today but first I'm going to go hit some more golf balls what did I say about this not being a Golf Channel sorry but yeah David gave me some coaching at the driving range and then it was back to our create week projects for our last day of working on them and naturally

likely because it was the last day this is also the day that we ran into our biggest problems with our project so by this point we had decided that we were using a GPT assistant and not one of the SAS tools we've been trying cuz we were hitting limits with those with the length of some of our training docs and we couldn't get a specific I guess when it came to training the AI how to respond because it was very interesting for me to learn that it's one thing to give an AI reference documentation but you also have to give it context on how it should use that documentation and how it should communicate with the user based on what you've trained it on we were asking Al to do specific things like ask follow-up questions respond in a tone of voice that sounded like kit give practical steps that the Creator could take next and we're working on getting the balance right because it' be sometimes that we would respond with like way too many steps or way too many questions and we also had a seriously hard time getting it to stop using exclamation marks so much even when Tom wrote do not use exclamation marks he used a freaking exclamation mark But while most of the group was focused on testing and adjusting the training instructions myself and cait so this is the creative director and the life cycle marketing manager we were coding I was building up that mockup of the interface inside kit as you saw and Caitlyn was working on our external facing demo site and we were feeling pretty proud of the way that we both managed to connect the chat GPT AI to our interfaces I was using replit Caitlyn was using chat jbt to Output HTML and CSS uh but then we hit two I'm putting out four fingers two major issues I think Caitlyn was just working on styling something and put something into the AI to like see what the looks like and it responded with this we were like hm does our AI have a sense of humor now what's going on and so Caitlyn said this and it responded emails now Caitlyn definitely does have a sense of humor so she followed up asking this and naturally our ai's response was emails I don't even know how to explain it cuz it's not that funny in my retelling but I absolutely losted at this I was literally crying laughing I think it's partly due to being tired but also partly because we' put two days of work Bren by this point and what we built was an AI that could only say emails honestly kind of fitting I guess for an email marketing software company but it wasn't what we were going for but while Caitlyn was dealing with that and with my crying laughing I was also dealing with another issue ever since that morning I'd been trying to deploy what I'd built in repet so that I could share the link around with folks for testing and it kept failing to deploy I kept trying to fix it feeding its own error messages back into it but I eventually got it into such a mess that I couldn't even roll it back to when it was working like I somehow even broke the roll back point and in the end after many hours of like fighting with this I did something that I really had been trying to avoid and that was start a again from scratch it was very frustrating but I guess it's kind of like a hackathon ride of Passage to have to do something like that it happens I rebuilt it and this time I just didn't try to deploy it we just used the like preview link instead which I didn't know existed in the first place for our last session we spent time on our presentation because did I mention at the beginning that there are actual prizes at stake here and then David and I went to play golf again we saw an even bigger crocodile this time 13 yards but this is the last time I'll mention golf in this video I promise we spent a bit more time on our create week project that evening as well there was just like some things we wanted to find tune about how the interfaces were working we wanted to put the presentation into like more of its final form and then it was officially the last day of our Retreat and it was presentation day it is 8:40 a. m. we

present at 9: running through the presentation just you know for the first or second time but it's under five minutes already but it's also because I'm like talking a mile a minute so Tom can go down to 1. 5 speed yeah I was honestly feeling really good about what we built until I started watching the other team's presentations one group had not only Built apps for the kit App Store using AI but they' also made a bunch of training videos to help other people do that too and I'm just like the sheer amount of work that they did another group made a prototype feature for Kit where you can save your brand fonts alongside your saved colors and apply your brand templates with just one click it was really cool that was the one I personally was most excited about another ships like I don't even know how many accessibility improvements like actually live to the app during those 3 days very impressive our presentation was in the second session and by the time we presented I didn't think we'd have a chance at winning any of the prizes but I did feel very confident that we had the best walk up song EMS EMS instead of telling you how I presentation went how about I display it for you so you can see what we built so we wondered what if every Creator had a growth manager and what if every growth manager was Nathan introducing the Creator coach AI so we train an AI on everything from podcasts to case studies to 400 knowledge-based articles to create a virtual CGM who could operate as a 247 coach creative coach knows the tactics and the strategies that build a valuable business and then knows how to implement those strategies directly inside of Kit as well we told the AI to act as Nathan Barry but to keep that a secret let's see what the Creator coach has to say So based on all the knowledge that we have shared with it provides some pretty good advice right now it's a bit generic and it's we can give it a little more context about Charlie's business specifically and the advice gets to be more relevant to her so suggesting that she could bundle some of her products and give exclusive workshops and if Charlie wants to know exactly how to implement this in kits the Creator coach will walk her through it but what if Charlie could just go open up kit hit command K and begin chatting with the Creator coach who has the full context of her business and all of her kit data considering her open rate as it gives her a strategy and knowing exactly which digital products she has available that she can offer her subscribers this is much closer to having a CGM that has the full context of your business and is going to be able to give you more relevant strategies and ways to implement them so meet your creator business coach while the judges were deliberating

I'd been asked to run an activity that we did at a retreat back in 2019 where we dreamed really big about all of the creators that we wanted to see using our product so we had everyone put stickies on the wall of creators they admire creators who they want to be part of the kit Creator Network creators who we want to get as customers in the next 5 years and it was so much fun to do all this dreaming together we had like a way bigger board than last time CU our team is way bigger than last time and now we have this like customer vision board that's like shared amongst all of us then it was time for judging and my team won nothing we won nothing just as I expected seriously though I am so proud of what we built I wish we'd had more time to work on like the marketing of our creative coach AI to show some examples of how we'd use it to reach creators and build brand awareness but I guess who knows maybe we will actually complete this project one day and release it and we'll get to work on the marketing of it then my biggest

learnings from this first hackathon though was well firstly there's so much fun definitely would do again but two that we probably should have assigned different Focus areas at the beginning of the project of what each person in our group would ultimately be responsible for I think we did really well planning as a team what we need to spend our time on but that maybe we could have gotten further with a project if we had broken it down even more individually instead of just having everyone pitch in to most things if that makes sense we had dinner and a bit of a dance party on the beach for our last night and it was just a really great end to a really great week I've been working at kit for more than 8 years now which I know is not the done thing in the tech world to stay at a company for that long but this is why I stay I'm still growing I'm still learning new things and I'm doing it alongside smart people who I genuinely enjoy being around and working with and it's just it's very special if you'd like to see what a regular work week in my life looks like when I'm not in Mexico dancing on a beach and driving golf carts around their crocodiles then you should check out my first creative director Diaries Vlog where I take you behind the scenes on some of the projects that I'm working on and show you what my work routine is like in a regular week so I'll see you over there EMS emails emails down