From Dev to MVP in Less Than 30 Days: Real-World Lessons from Databricks Engineers

Uh thanks a lot for your time. I would like to start with a huge thank you like generally for coming here today. I know today and yesterday if you've been attending yesterday as well was quite intense. I do hope that this session as all others that you have heard today and yesterday would be helpful to you in your endeavors, business endeavors, technical endeavors, whatever is that you're doing. And I do hope that this session as well will be helpful for you to accelerate your ideas whether it's just something small or something already in progress. So here's what we're going to speak about today. So I'll tell you about Genai photo booth that we at data bricks host just over there. You might have seen this already yesterday or today. If you haven't, please drop by. We are really happy to see you all there. Um we will be speaking about how we develop this but not from a technical perspective. I'll set this expectation straight away because uh technicals could cost you another two hours session at the very least and we value time. Uh we would be rather speaking about how exactly we at data bricks approach MVPs with our customers. How do we plan those? What do we look for? How do we shape this and promote this with the teams? I will speak about a bit uh on the internals and the externals that you should also keep in mind as we do and just close on a small note on what you can do uh starting basically next week. So um just a bit about me like uh a bit how and why I'm standing in front of you here. So my name is Daria. Nice to meet you all. I'm a solutions architect with data bricks. support mostly startups and digital native businesses here in the UK. Um we what we do as solutions architects as engineers we help our customers adopt the platform in the most effective scalable optimized way that fits their architecture and their technical strategy. uh myself I'm coming from uh consulting experience so almost uh nine years supporting large banking institutions and now I'm in startups and you know what I can see uh people regardless of the scale of the organization is what they seek is how they can build something small but something valuable really fast and we at data bricks we are really passionate about just that helping our customers ports and uh help them build data ini products in a real fast scalable way. You might know us as inventors of layhouse paradigm. We are also huge fans of open source. If you are as well, you might know that our founders have created a lot of projects that are currently very popular in the open source world and those projects form the backbone of our approach. platform data platform that helps encers to all kinds of data personas uh whether you are an engineer or uh a data scientist or a data analyst or a business person as well. So and uh one of the things that we as solutions architects uh do with our customers is help them build those MVBs on the platform. So how do we as solutions architect do that? Uh there are a few rules of thumb that we usually follow when sort of choosing and scoping for an MVP. So first of all let's talk about like what constitutes a good solid idea for an MVP ji MVP or anything else. So first of all two questions equally important who will pay and who is going to use users and by users I could mean yourself as practitioners or business users as whoever is going to work off the back of whatever is that you're creating or anyone else they want to see the value of what they're paying for what they're investing their time with. Um second thing like we as solutions architects as engineers we usually try to listen for frictions in the business like for example if you hear uh questions like it's too costly it's too complex it's too expensive our ears like perk up immediately because that's the point where an idea could already start generating interest with the business. Third question. Of course, we are custom obsessed and we want the users again this disclaimer what user is. Uh we want to know how the users will interact with whatever is that they're we are building. Users won't want to adapt their workflows to the architecture. The architecture should cater to their workflows. And fourth point equally important. So we sometimes hear from our customers and just generally things like we want to try genai. Why we ask? because it's cool, because it's hype, but does it really solve the friction? If it's not, that's your red flag. Seek the excitement from users by even the most boring technological stack, but if it resolves the problem, you're in the right spot. So our story so Jennif so our marketing team has approached us uh saying they want to increase the engagement uh at off-site data bricks and data bricks events uh from that statement alone we could already see uh our importance points that help us shape and scope who is going to pay the marketing who is what is their uh golden metric what is that they're looking for the level of engagement say number of people coming to the booth uh what is our audience non non-technical or technical users but rather the ones who are just genuinely interested in data bricks and we also can see the point of friction right how to develop applications fast with data bricks how to develop pipelines so quick uh efficient workloads is a point of friction that we could uh target so that's how we came up with the idea we develop a geni full uh which and if you have tried this you have already heard that we take a headsh shot of you. We run this picture for a genai pipeline with your prompt of choice. Say you want to be a Harry Potter. So we attach that. We generate the image and print this as a small token of uh you know remembers of the good days. Uh but the thing is except the camera and the prints the rest is fully developed and hosted on data bricks using native tools available there. So that way we could als bring people to see us and talk to us and as well as speak about how that they could do the same on the platform. So again I uh did that disclaimer before for technical details I'm not going to bore you with that quite a big day uh anyway for technical anything. So for this I encourage you highly to read this article written by Chris and Gabrielle uh on behalf of whom standing I'm standing here today. So it contains boilerplate code instructions how to build the same thing except for the camera and the printer that you can see over at our booth. So that's our story just a brief on the architecture just so you can see how it looks like. So uh whenever we take a picture we load this instantly into UC volume uh which is again a part of the data bricks. We have chosen instant IT model as one of the best uh resulting models that we see from the latency and from just you know handling facial features and such. Uh that took us quite a lot experimenting with that as well. uh with MLflow again native tools available on data bricks we are able to load this to uh serve this as an endpoint with data bricks which we directly serve into the application that we host again on data bricks and application is just like any web page that you could see on the internet with people take uh picture taken picture generated and buttons to print so it's uh simplistic right it's uh it took us quite a few iterations to arrive at But the first question in this architecture is what is our core functionality? And this is the first question that you might need to answer yourself when developing an MVP. What is the core functionality? Because that would be the focus, the backbone of whatever is that you're building. And for us, it's simple. How to take the picture and put it into gen model and get some results. So that was the focus of our first week, right? how we can deliver basic pip pipeline and endpoints uh within a few days. So here is sort of a lesson number one that we took out of this and how we basically uh reiterate the same with our customers how we can ship something really quickly. It could be ugly but it works right uh and polish later. So everything that can be polished later and that was the focus of the next two weeks. hardening, polishing API permissions uh in Unity catalog uh choosing the right model. So we have hit this um issues with uh the model itself generating kind of weird images. So again natively on data bricks we were able to switch the model uh and due to our modular approach we were able to test that quickly as well and that's sort of the second point which is heavy testing right so uh we were able to achieve relevance and quality with offline evaluations and a lot of user feedback. So that's a lesson number two. As soon as you can and that's why you are building this so fast, bring people to try and give you solid unsolicited feedback. So that's sort of lesson two. Lesson number three because it's doesn't exist in a vacuum and we will speak about this a bit later. legal, security and finance. What helped us really is reiterating the whole MVP and the ideas for the future in terms of okay what what is going to happen after we develop this with our stakeholders. So we have been uh constantly on call with them for data usage, data protection, how we can uh and can we even store pictures of people uh spoiler we can't and we don't. So the picture is deleted once it's printed. Uh JDPR and stuff also we'll touch on that later. We revised sizing a lot. So how we can introduce rate limits for example how what kind of hard guard rails we can have in order not to overspend the budget that we've given. Uh talk to them how we can offset the repeatable cost. So these are the things that are sort of throughout the development of the MVP. It's highly important and it's not point number three to keep your eye on this h on these things at all times and you can see by the end of week three we have done everything. So we have deliberately snatched 25% of the start to develop the bas the basic the core functionality and 25% of the time at the end in order to

test this heavily for only late minute issues. Again, MVP doesn't require you to have an ideal polished version of whatever is that you're building immediately. It's not the point. The point is deliver minimum viable product that is maximally engaging with your users. So here's our plan for you. It could be not week one, two, three, but sprint one, two, three. You could have instead of a week, you could have a month. you could generate, you could work on a gen AI pipeline or a data engineering pipeline or you're just doing a pet project. But this is sort of things that we uh kind of work around in terms of a framework when we're working with our customers as well. Having so having touched that in terms how we plan it right now what happens uh in between how do we make the right impression people aware of what we are doing and how most importantly we speak to the business. Uh the first thing so we solutioners practitioners we are nerds we like to really be proud of reducing the latency by 20%. uh automating 50% of the workloads. Uh small numbers that you can currently use, you can see yes it it's working faster and you and the business might even adhere to that. They might even be sympathetic to that. But in the end of the day, they will ask you so what and it's very important to speak to the business with their language. This is sort of a cheat sheet uh that I have created in order to illustrate that regardless of what is that you're doing trying to optimize make better all of those metrics could be mapped onto three things that the business really cares about it's either time or revenue or risk so by instead for example we have uh deployed a new model which is less uh hallucinating for your I don't know customer support pipeline. You could say you have avoided risk uh with compliance and brands because your agent is now I don't know speaking better and referencing the right uh documentation instead of data freshness you can say time to insight right you're delivering your data quicker to your business and so on. So this is sort of and of course one of the good part if you can uh transfer those metrics into actual money you're in a real good spot because in the end uh this is something to talk about at all times. Now that's for the business and but that's not just it. We humans we want to be heard and known right. So we want the rest of the teams know what exactly is that we're building. So a few tactics that we've used in order to promote this whole uh MVP and tactics that we employ when working with the customers as well. So uh strong narrative is the point number one. So you should already have that if you've worked through the whole uh planning exercise. What we also did is we shared one slide is we just prepared like simplistic slide deck not really much of information in terms of details just exactly what it's doing what problem does it solve how it works really simple byes that we could share with heads of something um of different departments that they could share it down side uh to their teams and promote your MVP result uh again uh value the time of the people who you are talking to right don't ask for huge uh hour lengthy unless they ask um meetings with them in order to explain that what you're doing uh also helpful thing that we did just demo we just recorded like literally one minute demo here's where you click here's what's the output and send them across together with the one sliders uh reporting so for the wider audience we have created a slack channel hey guys here's what we've done this week, here's what our plan next week. Also helps increase awareness and I touched on this already, but doesn't hurt to touch again. Reiterate with the stakeholders that really helped us not just be boastful and and good about what we've done, but also responsible about what we are doing. Um, a bit on the rest that's happening, right? So the pitfalls some of the pitfalls so that you don't have to hit the same I've put them into two buckets and the first bucket is going to be the internal one right so regardless of the industry that you're in regardless of what your company is doing role is it all boils down to basically the same components when it comes to developing a product and the first one is the stakeholders right widely used word especally actually today uh those guys will be asking you how you're going to measure success right so uh finding the right metrics for your MVP for us again it was engagement right but for you it could be anything else finding the right metrics is the most important thing when speaking to them because that's how they will say yes your MVP works we want to expand it uh also if you're from consulting as me rai matrix so responsible accountable able consulted and informed also helps to sort of navigate who you are going to speak to when promoting an MVP. Second pillar finance those people will always be asking you how do you spend my money? How do you what kind of return you could offer me and here and what kind of uh plan you have going forward. So for us it was really helpful to agree hard guard rails with a finance team. So if the budget is overspent we kill it or if the rate limit is that we kill it. So it uh helps us to ensure to gain their assurance that we know what we are doing. The third pillar is legal and security and they are in the end concerned about one thing. How are you protecting the data? Right? We live in a world where data protection, data privacy is not just highly important thing but also uh legally enforced. So having an answer of how do you work for every security thing? Having to review those security uh internals with your actual teams helps reduce risk with the MVP. So that's the for internal bucket. Uh helpful thing to do at all times log blockers and risks because when you will be reiterating that post MVP that will help you shape the future of the product external ones. Right. So where we are at uh at the end of this presentation right uh we know what the business wants right they want happy customers positive return on investment they want recognizable brand they want to derisk it and we have already gone through all of these ordeal in order to deliver stuff we spoke to stakeholders we gained their attention we raised the awareness we g the right metrics and found them exactly what where we want them so in the end of the day does it give uh does it give the business what they want? And the answer of course is yes because you've done all this great work. But again, we live in a world where there are things much bigger than us, much bigger than your role, organization. This is just an excerpt of the things that we usually have to speak to uh our customers about uh when working out some new feature with them, some new use case, right? JDPR uh HPA if you're working in healthcare um a few of the legislations if you're working with the US market or any other market right so this is just a selection of things that should be on constantly on your background in order to ensure you're not missing everything because the fines for this are extremely enormous it's not doesn't mean that you're going to pay for them but somebody will and that is painful and of course apart from the uh legislation itself any content rights, any data processing uh rules, any third party tsns. If you have a portion of your MVP of your product that is reusing some proprietary software, check this. This is extremely important to uh figure out before you start. So in terms of recapping, so a few of the key takeaways. So uh first and foremost, right, we spoke about frictions, we spoke about business, what the business wants. Look for the frictions. Look what causes them pain. Don't really don't try to test the tech stack because it's cool, because it's new, because that doesn't shoot an MVP if it doesn't resolve a problem and have measurable metrics because that is going to help you navigate what exactly is that you're going to do. Shape the core functionality first like right uh the walking skeleton something that's ugly but it works and that you will be building around in the future. You can always polish this later. I know for myself uh having to sit not even having uh being tempted to sit six hours through the night in order to polish this exact SQL query because I find it cool. It's amazing, but it doesn't solve the MVP problem. Translate tech wins your technical objectives into the business objectives. So again, time, revenue, and risk, those are the things that you can tailor your MVP to in order to be more relevant to the wider business. Uh, always have a strong narrative and always have a quick bites for promoting your project. It could be just that one slide deck that you have just save saved on your phone that you can bring in a sort of an elevator pitch or have you heard we're building that or this is exactly the problem that we are currently solving with our MVP at separate department something that just helps you to raise awareness quickly and efficiently regular checkups like catering to your health you should be catering to finance legal and security at all times and this is this goes without saying that those uh have the power to shut it down whenever they can and you wouldn't want that because you've already spent so much time and passion and energy in building what you're building and of course MVP the last letter of MVP it is a product so all production rules apply any regulations legislations company policies third party terms and conditions all those apply even that's a small thing that you have automated for yourself or your for your smaller teams So be aware of that what you can do next week right uh if you already have an idea or if you are already working on something that's something that you can basically start with pick one use case something that is really small and reuse everything that's available for you uh for example in data bricks we are offering solutions accelerators and sample notebooks which I have referred to in the article before uh to build something quickly to test your idea and see how it flies. Send this to your stakeholder with a short narrative. Here's what we're doing. Here is how we can help you analyze. It could save you this much. They're like, "Wow. " So, you could try that uh and see if it flies. In case of any questions, we are over there at booth M80. So, it's a bit inverted. So, we are here. M80 is straight ahead. The photo booth itself is back there at M1 112. Uh, here is Odai. He's amazing. He's back at the booth and can also answer your questions. I will be there as well in order to speak to you if you want. So, thanks a lot.