What a Billion Database Rows Look Like in Real Life

If you could take all the data inside of a database and make it real, how big would it actually be? And I went back to the very first data storage we ever had, paper. This right here is 50 rows of a database on a single page of paper. But of course, that's an absolute nothing burger. So, let's raise the stakes a little bit.

Now, this right here is a thousand rows of data on pages. Now it seems like nothing but just understand this for a second that a whole entire database I can hold in a my hand like this. A thousand users for example is nothing to bat an eye at. If I wanted to find someone named John in a database for example doing this in Postgress would do it in like 10 milliseconds maybe even less. trying to find it here might take like 20 seconds if I am doing it all in alphabetical order. And like to be honest, I didn't even try because I just know it's going to be slow and I got things to do. At the end of the day, paper stores data, but it's not really good for retrieving data. I don't know why I'm trying to convince you of this. And for 10,000 rows, we have over 200 pages of paper, which seems like a lot, but think about 10,000 rows for a second. That's a decent size application if it's users, for example. Now, 200 pages is cool. But um if I wanted to do like a 100,000 rows for example, that would cost me a small fortune in paper, which a I don't really want to do or spend the money or b I mean it's just not good for the environment, is it?

So I flexed some of my VFX skills. If any of you say what prompt I used, I'm going to kill you. 100,000 rows. That's like 2,000 pages of paper, which again doesn't seem like much at the moment. And 100,000 like users, say, that's obviously just one table. This is not how databases work for the most part. But what happens when we have over a million rows, which you know, we could add a pet table and get that pretty easily, especially with 100,000 users. Well, that would equal to 20,000 pages, as you can see right here. basically the same size as me, 2 meters tall. This is a lot like the picture of Margaret Hamilton uh standing next to the moon code or whatever. Um except for me, it's not as impressive. I mean, or really impressive at all. Now, in a database, this gets queried instantly. You don't have to do any waiting whatsoever. This probably happens so many times per second. And these two stacks aren't just sitting next to each other for the fun of it. If I wanted to find out which pet belongs to which person, I'd have to grab a pet, check the user ID, come back here.

Yeah. Yeah, I give up. Well, this is what's called a join. Databases do it in milliseconds. I do it in well, never. I'm never finishing. On paper, yeah, you just need a lot of time. And joins are one problem, but here's another. What if I just need to find one specific user in this giant stack? Well, this is where indexes come in. An index is basically a second skinny stack, just the names in order with page numbers pointing back to the original. Smaller than the main stack, way faster to flip through. Now, imagine adding sticky notes for letter ranges A through E, F through J, all the way down. I've got the alphabet memorized. Hopefully you do too. So when someone asked me to find John? Well, remember when I said this would take like 20 seconds before? Well, it's done now. And that's an index. And real databases don't stop at one. You can have indexes on emails, on dates, on anything you query a lot. That's just how Postgress pulls one row out of a billion in milliseconds.

Okay, 100 million rows. How tall will that be? Well, the database chugs a little, as it should. Queries take a little bit longer, especially when you're doing this. With paper, that's 2 million sheets. So, in here, well, that's like 20 stacks of paper my height. Yeah. Uh, I can't even walk in here anymore. That's 10 tons of paper and almost $200,000 in printer ink alone, which I didn't even factor in this, to be honest. And that's significantly more than my entire camera kit. VFX saved my life here. But if I was to stack it up all in one, well, we're working roughly to the height of the Eiffel Tower. All fitting inside of a single database.

And this isn't some sort of magic box. It's Postgress, the goat of databases, which you can easily use with the sponsor of today's video, Superbase. You've already heard about them, but Superbase helps you build your project in days rather than weeks with everything you need for a backend like Postgress, authentication, edge functions, storage, and real-time subscriptions. You can use Superbase with basically any framework you or your AI agent decides for you, and Superbase scales with you. Some of my favorite features is branching of your entire project or read replicas for faster speeds to all of your users. To get started, you can go to the Superbase dashboard. Or if you really want to go crazy, download the MCP server and just get your agent to do it all for you.

Thanks to Superbase for sponsoring today's video. So, what about a billion rows of data? You know, as if it was on one table or something. Well, taller than any building in Canada. That's what a billion rows looks like when you pull it out of the machine and make it real. 20 million sheets of paper. that's 2 km tall. And of course, we could argue about trillions and whatever, but I thought, why don't we take some real life use cases and stack it up, see how that goes. So, at this scale, companies aren't just storing data, they're drinking from a fire hose, essentially. So, what does their numbers look like on paper? Spotify generates one trillion events every single day. every play, every skip, every search, most of that flows through streaming pipelines, not a single database. But even if you take that same idea of like logs or something being a single row, well, it still adds up pretty quickly. And I did the math. Printed that tax passes the International Space Station every single day. Uber ingests over, let's see here, 6 trillion rows of data per day across almost 20,000 databases. So I ran the numbers on just one day of storage. That stack of paper reaches from Toronto to Tokyo and tomorrow they do it all again. Now if we have something like Facebook, they have over 3 billion users. They trillions of interactions. their data warehouses passed 300 pabytes years ago. I mean, and if I try and calculate this one, you're past the moon at this point. I just I couldn't even figure it out because it's just so much data. But imagine all of that just one day at a time.

time. I start off with a question, how big is your data if you make it something physical? And honestly, the answer is pretty surprising. A thousand rows fits in my hand. A billion towers over my house. And the software that we're using every single day is generating space level heights almost every single day, which makes you really think like, wow, that's just insane. And despite us knowing how much data and information we're going through on a daily basis, that kind of scale makes us think like what crazy technology is able to retrieve the information at millisecond speeds. So next time you spin up an empty database, understand you're opening up a gigantic file cabinet that could be endless. And if you want to start your application with Postgress, make sure you use Supabase. It's a great platform. I use it for my clogged car video that you might have seen as well as almost everything. So, thank you Superbase for sponsoring today's video.