Data Engineer Career in 2026: Roles, Specializations, and What Companies Look for - Slawomir Tulski

Hi everyone. This week we are going to talk with Slavamir about the reality of data engineering in 2026. Slavamir spent most of his career at Meta. He's moved between individual contributor and manage management positions focusing on scaling data engineering support at the meta ads ranking system. He has also contributed to global hiring by conducting hundreds of interviews and helping shape the company's data engineering recruitment process. Now he works as an independent consultant helping businesses make most of their data engineering teams. On the personal side, Slovir is a dad a husband and he talks about being a failed bus player. I'm really curious about the last one cuz I — I am a failed drum player and the reason um — so I wanted I always wanted to have like a metal band — and um my friends and I so we were like my friend we were okay let's make a band. My friend is like, I'm playing the guitar. Then other friend, I'm playing bus. And I'm like, okay, I guess it means I'm playing drums. — So then I come to my parents, I say, hey, like I start the band. Uh, I need drums. And they're like, no way. — That's I went further into the journey. I actually have a bass. I just really bad at playing it. It's so much harder than data engineering. — Now that engineering is simple compared to bass playing. — Okay. — So that's why I failed. So I had a band in the past but yeah it was failed one as well. So — what genre? — So it was back in the school days. So we didn't talk even about genres like there was a guy playing guitar and there was a lady singing. So that was more like a pop but I just wanted to play anything. — Okay. So it wasn't like be or anything like — No, unfortunately not. I would love to do that but yeah — probably the most famous Polish band I think. Right. — Yeah. Could be. — Do you know them? — Oh, yeah. Of course. I'm — Yeah, everyone in Pol knows them, right? — Uh yeah. Yeah, it's — they are quite — It's interesting band for that country, especially the background. Yeah. — Okay. So, uh well, we hear um not to talk about your bus um career, bus player career, could be interesting, but more about data engineering. But u first of all I want to welcome you to um our interview. That's really nice to have you here. — My pleasure. Thank you very much for inviting me. — Yeah. So usually always in all our interviews we start I start by asking about your career so far. So can you tell us about your career journey? — Yeah. I will try to be brief here. There's a lot of written already on the introduction. So pretty much I started in the academia. So I was a researcher and I almost started my PhD before quickly realizing that you know I need money and I won't make money that way. So I dropped my assistant professor — and I dropped my research. — Yeah. So that was remote sensing and geosciences. It's pretty much satellites. So getting the satellite data and doing things with that. So I — it's like internet of things kind of stuff, right? No, no. It's more like measuring natural phenomenas with the satellites. So I supposed to measure the movement of glaciers for example or I was doing the forests measurement from the satellites imagery. So those kind of topics. So really you know close to the life sciences. — Pretty interesting. — Oh it was amazing. We were actually doing machine learning before that was the thing. So — Mhm. — but I dropped that because I need money and you probably know that university is not the most generous when it comes to money. So I dropped it and became software engineer. So I moved to London. I joined bunch of startups and I was software engineer gravitated toward data engineering because you know it's software engineering plus data kind of. So that was a natural match. I became data engineers and after startups I joined meta and at meta I was data engineer and then you know they were Facebook at the time I joined so I was one of the leading data engineers there founding data engineers and then moved as they moved to met I also moved to management and leadership uh where I was there — and yeah recently as you mentioned I'm independent consultant so that was like a personal decision to you know leave the UK and I was as I was thinking how to do it properly you know I also decided that I will try to be independent uh together with that move. — Mhm. And Poland is a very good country. I lived there. — Yeah, — it's amazing. — I was choosing between Spain and Poland. So, I was traveling across Spain and then I was trying to choose between these. — Okay. So, why how did how come Poland won? — Yeah, there's a lot of family background here. know this, but honestly it's like choosing the rational mind tells you Poland like — and the heart and the weather tells you Spain. So nothing can beat Spanish weather and kind of chill. But yeah, there are some other things than the weather I have to take into account. — Mhm. Yeah, Polish economy is really has been on the growth recently like recent 5 10 years. It's really nice to see that. — It's also good to have smaller taxes. I get used to the British taxes. So I was paying effective rate at some point 30 46%. So almost half of my salary goes to — that hurts. Like I am in Germany so that hurts. — Yeah. So you know the thing yeah is better with that. Yeah. — So when you started working um was data engineering a thing back then like or was just everyone was software engineers? — Oh that was early days of data engineering. So the term data engineering was coined I think just shortly before I switched to software engineering. So I started as a software engineer but the data engineering was already a little bit of a thing. — Mhm. What was the state of data engineering back then like everyone was talking about Hadoop I guess. — Yeah that was the time where the Hadoop was becoming the thing. So everyone were saying that big data will change the world. So we have no AI. There was like big data supposed to change the world and we have just a data warehouses and Hadoop was that was the time where there were like so many consultants doing Hadoops and every single company regardless the size would go to Hadoop just because everyone needs Hadoop. Right now everyone needs AI, right? Back in the days it was all Hadoop, right? — Big data was the AI of these days. — Yeah. you have 10 GB of data like let's do Hadoop cluster for — so that was the state of data — I remember the pain of setting up a Hadoop cluster I was um yeah not something I miss — and there is a reason why we don't see Hadoop anymore right

— yeah um but uh so first you worked at startups and then at meta and I assume in meta in Facebook — uh you have a lot of internal stuff right I you don't need to talk about meta only like things you can disclose but I imagine that um meta — is known as all the big tech companies are known for implementing reinventing the wheel right — yeah yeah so that the like any single technology uh has either open source equivalent at meta like presto right — or it has kind of it's I would say counterpart which is internal like data swarm would be the example so nowadays industry is running dbt for transformation. So if you're building pipelines and you have transformation like most of the folks will use dbt for that at meta you have framework called datas swarm. Uh it was once it's not like a it's a public information data swarm was described by some articles so they are open about that. So, so we do that as one there. Well, we used at least — uh but honestly it's not a problem. Those tools are at the end of the day it's just a different sometimes syntax sometimes different formula but mostly it's the same thing. — Mhm. And since you've been working or observing this role cuz later in your career you switched to management but I believe you were still exposed to data engineering. — Yeah. like what has changed like do we still as the industry do we still do we agree on the definition of a data engineer or it's still different — no we don't and that's the big problem and I think you know I think the biggest game changer for me was actually leaving Meta and before that starting social media presence that was the moment two big moments when I realized how different my data engineering perspective is than others and people can't agree on this. The role is more than decade but uh people can't really agree beyond the basics right so everyone will agree that data engineer probably build some data platforms integrates the data so bringing the data into the platform transforming the data and exposing it to user right so ingestion uh extraction transformation loading and all of these things so building pipelines we're going to agree here but there going to be such a wide variety of actual responsibil actual task between the companies and different environments and it's a little bit for me it's like saying I'm a software engineer like software what you do you build website I mean no one builds website nowadays but are you front end are you back end are you this are you that what the hell like if you tell me you're a software engineer you probably build code build things with code right you probably coding and that's the same for data engineers if you're data engineers you probably do something with data — but that doesn't tells you that much and I think that's a big problem and you know I had constant uh you know discussions around like oh data engineers are doing this or they don't not doing this they build dashboards no they don't build dashboards they build data models no they build pipelines no they are not only the pipelines they are building business capabilities oh no that's analytics engineers so there is mess and uh yeah it's interesting that despite their role being so long you know decade is not a it's more than a decade and it's still there's a lot of vagueness and useless discussion. — Mhm. So when somebody says I'm a front- end engineer it's kind of clear that they are probably working with react or view or like some sort of — right but with data engineer you say it's still not clear. It could be maybe I'm just uh using fiverran and fan is taking care of everything or maybe I'm writing uh everything from scratch and like I'm using tafka and like it's really low level. — Yeah. Exactly. That's the point. Yeah. — Have you observed any how to say clusters like of types of data engineers? — Yeah. I So I my mental model is there are two of these. Of course there's like a generalist with doing anything like a full stack, right? But I think the biggest two clusters or the buckets would be for me what's something I called platform data engineer and product data engineer. So think about this again going back to the software engineering analogy as a front end and back end engineer. So the platform folks those would be strongly technical software engineering like skill set which builds platform. They build data warehouses, they build platform, they care about the infrastructure, they are good at DevOps, infrastructure, system design, you know, they pretty much build the entire ecosystem, right? Uh so that would be one bucket, things around that part. And then the other bucket would be more on the product side. So you already have the data is there in the platform, but business needs to do something with that because it's not surprised that you know by itself data is nothing, right? If it just sits there in the cloud, that's just a storage cost, right? You need to do something with data and all the kind of tasks and

responsibilities of doing something with that data. That would be the second bucket of data engineering which is more you know working closely with data scientists, analysts, uh product owners and trying to you know uh build actual you know analytical capabilities. So let's call it like this with the data. So that's kind of the different than building the platform itself. So those would be two biggest and let's call it buckets or specialization within data engineering. — When it comes to the t team setup or organizational setup, would you say that platform engineers typically work as one single team and then uh product data engineers are embedded in product teams? — It depends right. So it depends from the organization structure. Uh usually like from my experience when you have like a lean startups you would probably have just you know uh you know all hands on deck data engineers do full stack they are more like a generalist. When you think about the more mature companies, uh they t they tend to split those and you have like a platform team or whatever which will have those those platform data engineers and then you will have all the rest and internally they will probably even have the same titles. So all of them will be data engineers and uh their dayto-day is vastly different. Their skill set is different. Their dayto-day is different to sometimes to the crazy degree. Yeah, I'm just taking a note for myself. So, because I want to ask you about this. So, their skill set is different and day-to-day is different. So, can you maybe describe? — So, let's start with product data engineers. So, what kind of skills they need and what do they do day-to-day? — Yeah. So you know what do they do dayto-day is I will talk about theoretical perfect scenario because what they do sometimes can be sad reality but in theory the skill set you need is definitely SQL because you're going to interact a lot uh with the data through — I guess that's for everyone like if you're engineer you have — yeah pretty much yes so that's kind of generic uh some analytical skills some business sense kind of BI kind of skills so you're able to do basic dimensional modeling link analysis you're able to do a dashboard uh you can do transformation data transformation with dbt you can orchestrate your jobs with airflow you know those kind of things so you take care for the transformation uh building models building KPIs dashboarding probably and you talk to the business and you try to help them to uncover certain things right that would be your more on the product — how is it different from analytics engineering I don't see a difference and that's a part of the confusion right because what happened in the industry is there they like analytics engineers is how like 3 years old five years old probably not longer than that I mean depending how much — maybe five yeah — uh but they entered because they had the gap for to describe what they're doing because they were f failing under the data engineering — mh — and they felt like they are not platform folks. — Mhm. So because the data engine is you know the they would fall under the data engine which was more like okay we do platform here you know they started doing this analytics engineers but like there is many companies who didn't adopted analytics engineering right and they still frame this as a data engineer and I actually have those discussion like how data engineers different from analytics engineers and it's like — like honestly it's just um attempt — to break down this umbrella term which is data engineer and I think the long run in the industry will adopt that the data engineers will become the platform solely. — Mhm. — And then like this product side it will be taken away from the analytics engineers. It is sticks with us — but right now we are in the period where it's definitely not separated yet. It really depends on the company. Analytics engineers is still I would say rather niche. they are getting traction but uh there are still many times data engineers doing analytics engineering yeah — and I don't see much of a difference if you ask me you know we either make clear data engineering and make specialization or we break data engineers and we you know we frame as analytics engineers versus platform data engineering yeah we need to go either way because otherwise it's confusing — it's like um in Poland if you live on the west and east you use different word for potatoes, right? But at the end it's potatoes. — Could be. Uh I don't know. I told you I'm fake Polish, right? Could be. — I mean, so in one case it's cuz there's strong influence from Germany. The word is German. Yeah. Could be. — On the other side, uh it's like Polish word, but at the end it's just potato, right? So you cook it and you eat it in the same way. — But that's a big problem. You've mentioned analytics engineers here, but there are not only one ones in the mix. You have BI developers. you have still there are some leftovers of ETL developers there are data architects uh and how are they different than than analytics engineers right so — yeah and there are some companies and especially like you were we talk about that there's big difference between tech native companies and something I call tech by necessity

companies which is like the companies who have the tech departments just because they operate at the world's you know international or global level and they just need technical departments but there's a difference between these and sometimes they fra frame things differently I would say that outside tech they are more old school and they will have more of those classic — roles you find ETL developers there right — yeah I've worked with the clients which we still have the enterprise architects data architects ETL developers and business analyst — informatica and things like Uh they were on the SAP SAP. — Okay. — But it's not far away from Yeah. Um but pretty much like they were SAP heavily and all the integrations were SQL server and then yeah bunch of other things. — Would you recommend somebody? So we have a course data engine course and from what you describe we lean more towards tech native — stack rather than tech by necessity. But if somebody so some students who uh now take our course and then later they look for a job and they find a job um I don't know at a bank or insurance company. — Yeah. — Um would you recommend them joining this company as their first job? I mean, so now the the big question is like let's let's forget about the market conditions right now, right? Because for some people getting a job nowadays is a big thing for me is this. Uh as long as you're happy with what you do and you have the career growth path and you have space to breathe, you know, stay join the company, stay with them. Uh but if you join a company and that's probably like a old classic corporation or hardcore bank I would be really careful about what are your role responsibilities and how long you should stay there because there is high chance that you will became a in the machine where you do very narrow things for a long time and you know sometimes with some of my clients I remember talking to data engineers who just do DBT models for two years like two years of their career and throughout their career the only thing they would do they would sit — wait for the data analysts yeah that's I mean — I mean — well imagine if they had to use something like this SSIS integration service from Microsoft — in some sense yes but in the other sense it's the same thing right there you would have like drag and dropity now they would have dbt but it's still the same all over the time, right? It's very narrow and you know that's going to you're going to be replaced anyway probably if you that's the only thing you do — right because if your job is to sit there and wait for some business forks data analyst or someone to tell you what to do how to do and then you implement it in DBT that role is dead end. So if someone completes accomplishes the cars, they manage to get a job and they find out themselves in the situation like that I would say you know push yourself start up scaling because it's fine for now for a year or so but that's a dead end and you need to do more than that otherwise that's a stagnation and replacement. M okay so let's talk also about this technative and uh also platform engineers we didn't talk about them so what are technative companies — yeah so the technative companies you know of course you know all the you know the big four Facebooks Google etc that's one thing but all the companies which pretty much the fact that they are technical technology company it's making their products better right so I would say deliver I mean at least that's what the food delivery was in the UK deliver I would call them tech native — so even though their operations is physical so there's a courier who is delivering like Uber maybe is also — Uber is another great example but the reason why they are they took over the world is because they use technology at a master level to you know to make their services better and those services can be physical but that's not a problem right — and now compare is to let's say Coca-Cola, right? Coca-Cola is you know it's they were there always there selling you know the beverages they are not using the tech to the extent like the reason they have the tech departments maybe nowadays they are pushing harder I don't want to be harsh on Coca-Cola I don't I haven't worked with them so maybe they're adv more advanced right now but you get the point those are the companies which came later in the game just because they have to use the technology and uh and that's how the world worked and I think they tend to be a little bit less, you know, agile and advanced many times and they tend to be have more management layers. They tend to use a little bit more legacy have code. — They tend to have the IT structure like I've worked with one client I remember who was like really classic corporation and they still did the waterfall. They didn't have the agile. They had the waterfall big planning big you know it's still there sometimes — but that would not happen in the tech technative company like you know any SAS platform or any Uber tool. — Yeah. Okay. So we talked about different types of companies different technologies they use we also talked about the two kinds of data engineers. So one is product engineer. There's a lot of overlap with uh the role of analytics engineer. So they focus the product engineers um product data engineers is um like business skills, analytical skills, right? So what about platform engineers? So they I assume the focus is more on the engineering side rather than on the business side. So what kind of skill sets skills do they need and what kind of um what is the day-to-day? — Yeah. So definitely one number one which is like very underrated is going to be any sort of DevOps skills — because they either going to have to work a lot with DevOps or they will have to put all the deployments and CI/CDs etc themselves if there is nothing there so DevOps — they platform — engineers right so they have to maintain the platform — they maintain the platform so that's number one the number two of course all this kind of system architecture kind of things so now once there warehouse, lakehouse, data lake, what kind of feature they come with, what kind of uh technologies, tools and what how the architecture look like. So all the data architecture things they need to know that as well. Uh so those are two the biggest one. The other one is this kind of mix of this but that's a cloud engineering right everything is in the cloud right now and you need to be able to — you know to operate within those environments. uh then you will have some vendor specific stuff right probably your company going to be on the snowflake or on the Azure datab bricks or whatever is there out there you need to have the exposure for this and the cloud providers that's part of the cloud engineering right that's kind of their uh the core and you need to add to that of course all the processing engines like sparks prestos whatever you use to you know compute to to transform not transform to you know to pretty much to run run your transformations those would be the core for me and if you have this you're well on the extra addition on the top of that which I think many times is missed h but gives

you like big competitive advantage as an employee is being cost aware so uh I would say that the big thing is that you're able to match your platform, the platform you're building and designing to what your company actually needs and doing it cost efficiently and being in the process you know later on when you manage being cost efficient and the reason for this is twofold. uh you know one cloud bills skyrocketing is is a common thing in the industry right now and people are not cost aware you know we have this this thing that our cloud is cheap storage is cheap but then we quickly realize it's not that cheap as you think if you — it does up right — yeah it adds up that's one thing and the second bit is trying to build this behemoth platform for a company which is not at that stage. That's another classic joke. You don't go there. It's a startup — and then you Yeah. Overengineer. Exactly. And the flavor of over engineering here is like okay. So we are now ready for real time and batch and we have this lakehouse thing. Now what we going to do with that? We going to inject uh we going to you know uh inest CSVs, — right? So amazing. I can now I have like this Ferrari kind of platform which costs a lot and you know we are not using this that's another common problem. So I would say that if you know all the skill set I described is there and you want to get the competitive advantage that those would be other things you need to look for. — Mhm. — Because it's is a disease I mean it's surprisingly common across the industry to do both of those mistakes. — Mhm. So when it comes to product to overengineering and to uh being pragmatic, what would you what would be your recommendation for startups like I'm not meta, I'm not uh — Google, I don't have the pabyte scale. — So I just need to be able for my analysts to create dashboards. So there's some data being generated. I want to capture this data and I want to uh let my analysts use this data. what kind of technologies — like how would you recommend approaching it? — Yeah. So the first of all don't spend millions right that's the big thing you don't have and you could do things like nowadays with the current you know technologies and compute you could even have a DB which crunch something which used to be crunched by uh you know multiple parallel processing systems right so you really could get like things like data uh DB you know, Spin Airflow, DBT, whatever, right? You could even go simpler and have just like a DBT plus DB plus some visualization and you probably can run far with that, — right? Add to the top of that, I don't know, I think superset is good open source nowadays. I don't remember that one. Uh, but you pretty much can get very far away even with the open sourced. Uh, and you don't need real time. you probably are super fine with daily batches or if you need to go below that that's fine but you don't really need you know expensive tooling you don't need expensive systems — because if you're just a startup and you're just starting over you need to mind yourself like you don't have that big data something which used to be big data it's no longer a big data line no one is even using big data anymore like it's a reminder from the past, right? And we can crunch data, you know, even on the single instance and you can go far away with that. But you don't want to spend too much time on setting up all of this, maintaining all of this because every single complication uh just you know it adds up like let's say let's have a real time. It's not simple problem to get the real time like you can have a cafka but then you know the real time processing it's different be piece than batch processing and do you really need a real time right now I mean I'm 99% sure that you don't need that right — like even if it's uh like let's say there is a need for close to realtime data you can just run it every five minutes right — exactly and the it's much simpler and if you run it every 5 minutes. This much smaller because the incremental is smaller. So, it's just fast enough and it just makes things so much simpler. But I've se I've seen companies pushing real time in so many strange ways. — Mhm. Okay. Yeah. Interesting. And uh yeah, and at what point like okay, I have this uh lightweight setup DBT duck DB supererset. At what point I need to consider you know these big names like platforms like snowflake like datab bricks uh do I need them at all maybe I implement the platform myself — I mean I would not implement platform

myself probably because you know like you know even if you're implementing it yourself with just open source you're still paying for engineer hours right so it still does have — the code will not cut it right — uh yeah probably cloud will not you know unless you're a social media influencer and anything is possible, right? But jokes aside, uh I would probably go at some point to a little bit more let's call it enterprise solution. But that's where the scale comes, right? If you have more and more data analysts and you have data scientists and you do bigger things, at some point you will hit the ceiling, right? The things will get start to get clunky. uh management will get a little bit harder and you know you're probably gonna miss a lot of good features. Uh so you will want to move at some point. — Uh but just don't move for the sake of of moving if things are working fine even if you're in your basic setup and you can get the business value because that's kind of which uh too often people miss. At the end of the day, it's all about the business value being able to provide, — right? Uh so if you're a startups which is scaling and then things go like know you have market feed uh you know the adoption is growing you start to hire more you start to scale all the other functions that's probably the moment you're going to also scale your data engineering and you start to think about moving the platform you know to something more uh enterprise grade before that I don't see the reason — okay — uh what usually happened though is this you have a this is not a problem for startups honestly unless they got like a huge funding it's usually the problem the other way around which is the scenario goes like this there is mature corporation they used to have some other departments covering for the platform like some IT and engineering and they were not really data engineers right they were maintaining the things And then you had bunch of BI developers with some vendors and things kind of worked but there is a money in the company and there is like big digital transformation coming right and they are saying okay now we going to have this top level data platform they have money so they have you know a lot of money to waste and they tend to go all in and they to go like okay now we're just going to do a massive transformation and I think that from my experience that's the moment where we sometimes failed to, you know, to put the scale where it should be — and we think we have big data and we have we need this or that and there are all those promises and there's like huge marketing machine behind all of this like the FOMO is really you think you're missing something if you're not running this. So I think that the risk is more on those companies which has money and want to go all in into the data and they think they now need to drop you know millions of dollars. Startups go from the other way around. They scale in luxury. They usually are safer. — Mhm. Maybe these companies they are just used to uh spending a lot of money on SAP and stuff similar stuff. — Oh yeah. I mean and they will hire Deoid or whoever there the one of the big four players and they will give them big check because they do big important work and — Yeah. That I would say the risk is more here. Yeah. — Mhm. Okay. What would you say about Spark? Like is Spark considered legacy? Would you use Spark for any new projects? — Uh people are still using Pispark, right? I would — Yeah, it's there. But like let's say you start a new thing like would you use it? — Well, if I start new thing, I don't know like you know honestly if I really need Spark and I need to set things up. I don't know what scale I would have to have, right? — I would go with either Presto or Doug DB or like anything simple. If I'm just starting out, — my mantra would be just make it as simple as possible. — Mhm. — So, so you know, both I mean I maybe I'm biased towards Presto because I was at Meta that's why people are not so generous towards the presto so much but DD is probably yeah — DB that's perfect solution, right? But any others — will do as much do. But yeah, I would say Spark is not legacy yet. I still see a lot of Spark. But — but both Spark and Presa, they still require all this big data uh core, let's say. — Yeah, that's yeah, you need some leftovers. — You need a team of people for this, for both of them, right? It's going to be hard. like if you're a team of one or two data engineers pushing that I would not go with neither I would go simpler or vendor solution which gives you tool you know — so I like spark I use spark a lot and I teach spark in our data engineering of course — but I think sometimes it's like first of all the cost of owning the cluster is huge — yes — then uh another thing with press it's the same thing but there are — like for example in AWS there is Athena uh which is nothing else but managed RA right so this is — this is convenient — right so I was just curious to know what is your opinion about spark because I recently spoke with another data engineer — and he confirmed that um like for them spark it's still there they still use it but for new stuff they wouldn't um necessarily use it — yeah so I think that the rather the question here is are do you buy or build right if you want to own your own on infrastructure if you have team of people and very specific reason why you want to do it that way with neither pressed or spark or whatever right — mh — u you can do it is still there people are doing this but you're 100% right the cost of management is higher than uh than you think right managing those clusters doing the things yourself if you just buy something I think on the AWS on GCP you have their own uh you know depending on the cloud provider you will have their solutions I would probably go with those solutions, especially that you know, you probably don't have a big team, right? So, yeah, if it's up to me and and I'm setting up something for I don't know, my startup, I would go with those vendors. Yeah, I would not try to — set up in Spark myself. — Mhm. Kafka, would you set up Kafka? So, — no. I mean, unless I really need I mean I Kafka is amazing. Don't get me wrong. If you really need real time, go with Kafka. But I I struggle to see many realtime use cases for the kind of classic advanced analytics even — fraud detection.

— Yeah, fraud detection. But now you're talking about very specific uh fintech. You're doing fraud detection. Yeah. Okay. Go real time. Uh you know, you have a transaction coming in and you want to check every single transaction. Yeah. That's a great use case. But how many startups are there doing the fault detection, right? — So not that many real time recommendation that's another use case, right? Let's say you want a dynamic pricing or you want something like super — uh dynamic then yes. But now the big question are we doing now the classic analytics or are we doing something in between of you know AI engineering machine learning and software engineering. — Yeah. It look sounds more like software engineering to me right. — Exactly. — Detection like uh or this recommendations. — Yeah. And they will involve that the analytics teams especially if the machine learning folks are on the analytics side in that given company. But when you think about just you know reporting dashboarding and analytics it's hard for me to find single case for real time. If we look more broadly advanced analytics, ranking, fraud detection, m I mean machine learning and AI. Okay, I start to see the point there. Uh but he we are going very specific here. And my claim is also like this. uh like if you can't if you don't know what is your revenue and if you don't have basic reporting basic analytics if you don't know what happened in the past is it really the time for you to invest in real time insights like dude you don't even have the insights from last week maybe real time insights is not the thing but if you need real time I would go with Kafka definitely Kafka is the industry standard there is a lot of uh you know things built around Kafka. Everything speaks Kafka. So if you need real time I would go with them. Yeah. — Okay. Um we have quite a few questions uh from u people who join us. Um so I want to start with these questions and one of the questions I want and this is something we briefly touched when I asked you about these traditional companies um and how is it good to work for them. you said um you started like oh let's set aside the market conditions for now. Yeah. Right. But like if we don't set aside the market conditions now like first of all what are these market conditions now? How tough it is now? — And especially for juniors like cuz in our courses we have a lot of people who are switching their careers. They are not necessarily juniors like but we also have those two. — Yeah. Um so they already have some background not always in software engineering but like they have some background now they want to switch careers. — How tough is the market for them right now? — Yeah. So I can tell you only the things which you know I because of my social media present I talk to a lot of people and many of people coming to me with the problems of getting a job right. So the problems I keep hearing is that it's rather tough. it's hard to get even through the CV screening. Uh so my understanding right now I mean of course you can see less job this posting so I see that myself I see less uh less junior roles I see less posting about the job and people report to me that it's really high like getting the interview itself is a big success not mentioning you know going through the interview right so my understanding right now every report I've seen every person I talked to is ra market right now is rather tough Right. Uh so that's why I set uh you know I'm setting this aside. Uh now there is a different I would treat it differently people who are already within the industry within data related position. Let's say you are data analyst and you want to switch to data engineering. you are way in a way better position than someone who is I

don't know I used to be an accountant and I know Excel and — the question is from a person who is a civil engineering civil engineer — civil engineering so civil engineering so the good thing will be that they are they at least have the technical background so that's a good thing — what is civil engineering I actually — I guess it's it's a building site construction you design and calculate all of these I think that's kind of something like that — so Google says it's a core engineering discipline focused on designing constructing and maintaining the built environment including structure like roads bridges dumps and — water so you are good at math you're good at physics you're engineer you're really engineer so that's a good point uh it's still a little bit weaker than actually being in the tech industry and having you know analyst software engineer or other role right uh but it's still way better than you know I don't know lawyer trying to get into data engineering and having zero even in engineering. So the tearing — the tearing for me would be like the person I don't know completely outside engineering trying to break in through I'm not jealous in that situation then you know people trying to get into with some engineering background robotics engineering math whatever it's going to be hard as well but it's better in the sense like they have the formal training and they know the — uh they are engineers so that's better and then I would say a little bit easier are for Folks who are already in that data industry they are data engineers they or data analysts software engineers sometimes move you know machine learning engineers usually not because people want to get into the machine and AI machine learning and AI but if you are there you know you have the slight advantage because like you have already uh job within the market so my advice here would be reuse your current role to extend so let's say like A classic example would be you're a data analyst and you want to move to data engineering. Why don't you use your current role and extend the responsibilities and instead of data engineer building a pipelines for you, you do it yourself and you ask your data engineer counterpart to you know just review your work or help me or ask use them. So you could do these things when you're already in the industry. So that's a big advantage. — I've seen uh multiple people in my previous company do exactly that. They were analysts. uh well analytics and data engineer it's very how to say close right — yes yeah — but I saw a few very successful cases where people did exactly what you said and now they are lead engineers — but now the the scenario of the folk someone outside the industry and they try get into my advice here would be uh you know try to get a job like I'm going to be very brutally pragmatic get a job like your journey like probably going to be longer than other people because — like you don't even know if you're more on the platform side or maybe you're going to be more on the product side. Maybe actually you're going to love analytics engineering kind of thing. So you're like you're just starting out. So get a job and then try to build from there. That would be my advice. uh unless you really know that I mean you from whatever reason you're passionate about platform uh you're building those data platforms then narrow down your search and narrow down your learnings towards this because there is no way you're going to learn everything end to end all the frameworks from both of the sides of the kind of data engineering there's just simply too much like there's so many of these and yeah companies are guilty of just throwing bunch of random crap out there at Oh, you need to know this and this or that. DBT, Airflow, superfa. Yeah. And then you go to job and you build a dashboard. It's like come on. — Yeah. So, so narrow down. You won't learn everything. And yeah, if you're outside the industry, it's going to be harder. It's doable. Uh but I would I personally would just try to get a job within the related title and keep this as an anchor and extend from there. By related title, you mean like data analyst? — Uh, I would go rather with data engineer. If I'm really desperate, I could go that way. But that depends really on the personal situation and how desperate I am. — Mhm. But is it like a actually a good strategy because also people come to me with this advice doesn't always uh apply to data engineers, could be engineers, ML engineers. — Yeah. The idea is that there is a stepping stone like a different role for which is easier to get in allegedly first and then you use this role you work there for a couple of years. — Yeah. and then transition to your target role. Like let's say for data engineers it could be — because we know if you're already a software engineer for software engineers it's way easier to transition to data right because they all they have all the skills they just need to specialize in data right for data analysts they are already very exposed to all the data stuff so for them the transition is also smooth — smoother right than for others. So would you recommend to actually use this strategy of first getting some other role? — Yeah. — Or go try to go directly. — I would um my answer would be it's not possible to answer it without knowing the person's situation because let's not forget what's your age, right? Do you have family? Like you only have so much time, right? if you need to learn new things — and if you have three kids and you're this all person who needs to provide for your family right it depends right — I would say this if I have to have stability and I need the role faster — and I have some time to you know push after hours or within the role but you know I'm also constrained I could advise that right because it's longer but there is this kind of transitory period you could manage your finances and you could manage the transition smoother that's possible I could take it into account but if you tell me like you know you are not desperate and you want to spend you have more time to spend on extra learning extra projects and doing this extra work then don't do it because it will take your time right you know if you can spend and one month just doing like super hardcore uh curriculum and no work. You just push hard and you learn everything. You go to your community, you learn your stuff, one month, two months you can go get ML job, right? — But uh if you at the same time you took a transitory role like you now half of your day is that role, right? And you will only have a certain — uh certain amount of time. So it's there are pros and cons and I would say look at your personal situation. If you can spend time learning, pushing harder and not distract yourself with transitory roles, don't do it. — Mhm. — But if your situation requires it is I don't know is industry far away from software or from and your background is completely different and and you need to have a job. I would consider that. Yeah. — Mhm. uh what I usually think about is that data engineering and data analytics are actually quite different right so at the end what data analysts do is very different and the things you need to learn for data analytics — are very different like SQL is common data understanding is common but the rest — could be like broad right — yeah so maybe like if your target role is data engineering it doesn't mean you'll enjoy being a data analyst — oh yeah you they suffer right I mean come on like there are two different roles because of the reason right — so that's why I'm really saying it really depends right if you know I know people who just want to get to tech or the data and if you have really honest discussion with them they're like dude right now like anything will be better from what I'm doing right now I don't care data engineer data analyst just give me data position right if you're in this situation that's a completely different discussion than you're a software engineer who has uh you know not only background but also has opinion about what they like and what they want to do. That's completely two different stories, two different people with uh and they should have different approach. — Mhm. And I see there's like five questions that are about the same thing. I'll group them. There's a lot of concern about um AI invasion.

invasion. — Yeah. So does it even make sense to try to become a data engineer now when AI can do all this for us? — That's a simple one. Yes. Uh yes. And the caveat is this. If your data engineering role is what we described earlier on as this kind of corporate factory-l like model where you sit and you implement DBT all day long, you should actually upskill because that's going to be taken away. There's many things AI will take away. uh you know there's uh the tasks which are right now done by data engineers which won't be done by data engineer because they are trivial tasks building that DBT models — you know there's text to SQL there's like kind of trivial answering trivial analytical questions there's triaging like there's the whole let's call it subg genre of data engine which is data ops so — triaging — uh triaging like so pipeline fails and you don't know why it failed And there are the position they sometimes call data ops which goes there and like oh this failed because that process failed so we now need to restart this. It's kind of like you know lumber fixer which goes there looks up for the it's sometimes it's — AI can do this probably right — yeah I mean AI can rerun the things can understand the war like sooner than later AI will be able even to push a PR with the fix for the failure right so those roles are doomed right — but doesn't mean that the other opportunities are not there like I mean platform won't build by itself And even when you build this platform and there's AI — you know the better AI is integrated with the platform the better for you right it needs context it needs all the semantics it needs understanding of your company your data ideally is well classified there's metadata you can rug it you can you know integrate your models you know you can do all of those things and it won't make by itself there uh so who will be doing all of these no data engineers most probably right if I'm integrating all the fancy agentic AI into my data platforms those going to be platform data engineers who are doing this it's not going to magically appear there right and then AI needs to work on something and the cleaner the data the well the more structured the data are there the less work AI needs to do to infer the better it will work — right I will give you a classic example uh I have companies which has uh you know integrated at the vendor solutions for AI to do conversational analytics and they are like what is the revenue and the AI tells them but the revenue number is wrong right — because the data sucks data modeling sucks uh like they have just a mess so you put this AI on the top of the mess someone needs to clean the mess so there's a lot of work for data engineers with the caveat that you need to be this kind of verticile data engineer and not a DBT monkey or not a triaging person who just checks what failed and restart click the restart button like this kind of data engineering yeah that's at risk yeah that won't be there — okay good to know but I guess it applies not just for data engineering — yeah that's applies to everyone right that's data engineers are not so special right that's applies to software engineers that applies to data scientists as well like the — the more trivial more repetitive if uh they're easier to automate it, right? — If as a data scientist, all you do is uh tune ex parameters. — Yeah. We don't have good news, right? — Yeah. Exactly. That's exactly the same situation. Yeah. — Mhm. Okay. So, a few questions about the interviewing process uh and hiring. You said getting interview these days can already mean success because you speak with people and people share their stories with you. So, how do I maximize my chances of getting an interview? Does it u should I improve my CV? And if yes, how what kind of projects I can work on? Do I need certificates? Um like what would your suggestion be? Like if I want if my chance now is to optimize my — like let's say criteria success right now is getting interviews. — Yeah. So definitely I mean like so I'm assuming you don't have solid network right because there is this one of the advices that you should network and your job will probably come from the network and I fully agree with this but the problem with networking is that you already had to network before — uh and if you now want to get data engine job and you never network your — first thing you need to do right start networking — so I would start networking if there is one career mistake I did is I started networking way too late. And I will give you the CL example uh like I got my Meta job because the person I knew really well went to Meta and they recommended me. When I switched out of Meta and my first consulting gigs, they were through my network. uh when I started you know I could have full-time job uh which was not even like two or three already which were not even posted on the website because I knew the people I knew the companies so if you have the network uh you're going to get your job from the network you're not you're you don't need to worry but most people don't network so for getting networking what you need to do now is you need to definitely polish your CV — and you need to start sending those uh CVs there to recruiters. — How they polish my CV? — So, first of all, the CV needs to be uh you know forgetting about you know formatting. It needs to be clean, nicely formatted, all of that. Uh I would definitely focus on the outcomes. What the hell are you doing? Right? So for example, if I screen CV and if you tell me that, you know, I know how to, you know, how to build pipelines, that's probably not that big of the competitive advantage. If you tell me that you proactively uh build something to reduce the cost or whatever, oh, that's already a big thing. — So how you can use your CV to show the traits companies are looking for?

— What if I haven't built a pipeline that reduced cost? What if I'm just doing my personal projects? How do I present them in such a way that it's attractive? — Yeah, first of all, that's already a good thing because many people don't do even personal projects. — But if you don't have real like the the tearing goes like this. Uh if you have real job experience and real examples, that's the strongest evidence. Then level down is your personal projects. Now they are not as good as the real examples from the work, but they still count a lot. and then the tutorials and just you know just kind of proving that you know you have some certification some diploma that's the weakest argument — so if you don't have real work go with the side projects and advertise them I would say don't get uh insecure about these right I mean if you don't believe they are strong enough and they are good enough to show your skills uh how someone else will feel about them like that so if you have a strong site project, present it as a side strong side project and you know put some marketing hat on your head and fight for yourself, right? So the mistake I see sometimes people doing this their personal projects, they're kind of almost uh apologizing for the fact that this is the side project. It's like yeah, it's just a side project I did. Like dude, like don't apologize for the fact that you spend time to actually do this side project, you know, show the best of this side project, right? So if you don't have real experience, show those side projects. Don't be shy on this and and not actually advertise this. It's not obvious. Not many people actually do this. Like what most people do is they watch tutorials and they, you know, repeat the things done in the tutorial. So if the tutorial was like how to build dashboard using this or that right they will build a dashboard which looks almost like the one from tutorial. Uh but if you have real site project which you you came out yourself you extended something yeah you could be proud of yourself and present it like that way. Uh so side projects definitely uh framing you know business impact framing what changed on your CV instead of you know describing what you did that's another thing uh run your CV through you know different people so they look at it and they see if they even understand it right many times you know I remember seeing once one of the CVs and I couldn't understand half of it because it has the aberrations and acronyms and I was like dude what the hell is that and after talking to the person we realized that this is great CV for that given industry because in that given industry everyone knows what's metrics XY Z right they were using the lingo from specific industry and probably if you want to get hired in that industry is okay CV if you know if you want to get something else like just make your CV understandable like I don't know what are those numbers are what are those acronyms So uh that's another thing and then you know just you know try to network at the same time try to push your CV try to apply at the same time and build as many side projects as you want and prove yourself and will it magically work? No probably you still have uphill battle but it's you know you're increasing your chances right — yeah do you have time for a few for one more question? — Yeah yeah let's go. Okay. So, um we talked about optimizing CV. What about um projects? How can we optimize the projects we choose because there is infinitely many projects I can pick up from? — Um do picking up some projects uh like will picking up some particular projects will be more beneficial for me like on which projects should I target if I want to get into tech companies? — Yeah. So now this is why I'm kind of talking also about data engineering and different specialization because the clearer you are with what you want the better for you right so let's say that I will give you my personal example I wanted you know I knew that I will have referral to meta and I knew that I need to have stronger data modeling skills for meta I was like heavily on the platform side and I didn't have much exposure to data modeling and I knew that there is a project coming in to uh remodel our data warehouse at my current company and I said like guys I'm going to do it with the external contractors because the externals were higher for that and I'm going to do it extra hours I mean I'm just taking it as an extra thing uh just pay for course for me and I will learn and I will just put extra hours so I was working more but they paid me for training and I had real exposure so that's amazing side projects to take on and I was working more but I was you know very clear about why I'm taking this project I was very clear what I want here if you are like ah I don't really know what to pick up probably you're not clear enough what you want to do and the what you can do is either pick anything and see if you like it if you don't like it switch to anything else uh or you know spend time figuring out what you want to do and now if you know what you want to do now let's say I want to go to data engineering. I'm not in data engineering. I want to go to the data engineering and I want to be on platform side. That's already quite specific and then you can see what kind of things you will be doing. You're going to be building data platforms. So obvious choice for your site project is to build a toy data platform. — What is a data platform? So data platform would be the entire ecosystem of uh tooling around processing, storing and using data. — So I have some incoming data and my goal is to have a dashboard. Yeah. So everything is yeah I would call it a data platform. So probably some people right now from the industry will shout at me that no data platform is just data bricks or it's just snowflake or whatever. for me is the entire ecosystem that you know integrates the data so brings data stores data you know translate data and expose ites to user — income data incoming data is CSV file — I misspoke I said income but this could be actually good example so let's say I get some data from my bank account — with uh all the transactions right so this is my CSV file and my goal is to have a dashboard board where I understand my spendings. So everything that is in between these things could be like all this processing could be this uh platform and if I use the example you had uh tag db dbt supererset to glue — exactly so this is already a small platform right — yeah exactly and it's like all the pieces you will do as a data engineer so

it is connecting to some sort of APIs or getting scraping the data that simulates you getting data from upstream systems you're going to have cleaning of data because probably whatever you get is going to be dirty and unusable. So you have to clean data, you need to have the data quality checks as well probably, right? If if you have that then you need to store it somewhere and in a nice form, right? So you probably want to do some data modeling. Another kind of check box, right? Uh you know it's it may cost you something if you're willing to pay some money, but if not, you know, that's still fine. Uh and then you want to do something with that. So you build this dashboards that's another piece and you can even simulate you know getting business value out of it. You can actually have some sort of analysis or some sort of use case behind it and you can have this end to end flow end to end project and that's really valuable. It touches every single point of uh data life cycle. Uh you're learning core skills. It's a big one and you're going to learn a lot and that's a great side project, right? M and um maybe if you're like me and every year you have to report taxes um I assume most of us have to do this right — especially like if you live in Germany — um so every year I have to go through the same process I need to go to my bank accounts multiple accounts right they're different — I need to understand okay this was my income because I need to declare my side income right and understand how much what were the transactions with this side income and where exactly it's coming from Right. So, and it if I had a system with transa with a report at the end, I think this would be like a perfect side project. — Yeah. It's amazing. — Like it's very simple and it solves my problem — and then it's a portfolio project. — Yeah. And one thing which I like in this project is you actually personally will benefit and care about this and I'm like just from simple human psychology, right? If you are if you have to force yourself to work on something which you don't give a think about or it's kind of very abstract, you probably will have less motivation to push hard on this. But if you're solving something for yourself or something you're passionate about, that's you will get this thrill of working on this and you going to you will want to do it, right? So it's also much easier to push those kind of projects. — Okay, cool. Well, I don't want to keep you longer. Um we still have a few questions but I think some of these questions are not really on topic for this discussion. You can ask these questions in our slack and I'll be happy to give an answer but with that I want to thank our guest. Thanks a lot for joining us today — for sharing all your experience with us. Thanks everyone for attending too. Uh I really enjoyed all the questions you had. It was really lovely discussions. Thanks Lamir for — thank you very much for having me here and yeah if anyone has any more questions they know how to reach me if they don't they just put my name on LinkedIn yes and my I'm a real person I always saying I do talk to people so I told you about networking right so I do network I do like networking so yeah if

you have questions reach me out — do you go to attend any meetups in gransk — I haven't found yet any uh I need to find what is the scene I'm still in Poland it's just my second month I think So I still don't know the scene. I don't know the things here. I still — in GK. — That's the plan. That's the plan to stay here longer. Uh — if somebody is from D and want to hang out. So — definitely uh anyone from Poland, reach me out. I don't know anything. So you will be of help for me as well. — Perogi, I really like Perogi. It's my favorite. — Dumplings are amazing, right? So yeah. — Okay. Thanks, Lamir. Thanks everyone. And uh yeah, was really nice chatting with you. — Thank you very much. My pleasure.