PostgresWorld Podcast EP01: Scaling PostgreSQL Without the Operational Overhead

Hey, folk, folks. Welcome to the inaugural Postgres World podcast. And today, we have a very special guest with the head of databases of Supabase. Why don't you introduce yourself? >> First of all, I am honored to be the inaugural guest. >> Well, thank you. >> My name is Bikti Sigredi. And what I do at Supabase is run the engineering teams that uh create our database services. So, that is Postgres, which is our OG service that Supabase provides. And Supervisor, which is the connection pooling that sits in front of our Postgres service. And also Multigress, which is our next-generation sharding solution. And OrioleDB, which is our next-generation storage engine for Postgres. >> Okay, so the Multigress that we had, and I forget his name now, but we had him on my other my company podcast earlier this year. And the Multigress stuff, that's not necessarily based on, but inspired by Vitess, right? >> That's correct. And I had worked on Vitess for 7 years before uh joining Supabase. And for a lot of that time, like 4 to 5 years, I was the tech lead for Vitess as well. >> Okay. Well, I mean, that obviously brings us to expertise. Um the one of the things I like about Supabase, now I'm I'm not a big cloud guy. I've been around way too long, and I just have control issues. Um but uh you know, every time we talk about Supabase, every time someone's a guest on one of my podcasts with it, every time we run the conference, all of it, um you have a a a real active developer advocacy advocacy / loyalty base. It's actually really surprising. I mean, we talk about Amazon, we've got Google. I mean, they're all long-term heavy sponsors of our conference. Um but they're it how can I put this? Because I don't want to insult them because they do good work, right? But there there's a certain Nobody goes gaga over Google. Lots of people go gaga over Superbase Superbase. And let me be fair, Google, I love you. You know I love you. We deal with you all the time. But nobody's going gaga over AWS either. Um and I I think a lot of that is because there's there's two whole generations now that have that exist in our adulthood where Google and AWS always existed. So, it's all it's not this bad. Don't get me wrong, it's nowhere near this bad, but it's kind of like Microsoft, right? Nobody goes gaga over Microsoft either cuz they're they're old companies. Right? I mean, they're just they're just there, right? I mean, you know, I'm going to And so, the developers don't get excited because developers by their nature like shiny things and like to reinvent things and and solve problems that have been solved for decades. I'm looking at you, Jason. Um and uh but that's what developers do. So, what is it as head of databases? You've said, you know, what you're in charge of. But what is the unique thing? What is the next experience that you're going to bring us? >> Mhm. Okay. Um >> [clears throat] >> So, that doesn't It's not just me that is working on the next experience. It's the whole company. Yes, right? And I think you put your finger on something that uh has actually been a big factor in Supabase's popularity, and that is a prioritization of developer experience. So, our user base, and audience if you will, has always firmly been developers. And we consider ourselves to be in the developer tooling space, even though the service that we provide is an integrated backend as a service. It's not just a database, it's also other things that you need in order to uh run a production quality service behind your applications. >> So, outside of some obvious things like I'm Oh, I'm sorry. I was going to My question is >> developer experience has been uh a focus of the team starting with the founders. And I think that came through, and that is what enabled us to build this community of millions of developers who love Supabase, go gaga over Supabase in your words. >> Well, okay. So, that's interesting because, you know, there was another service once upon a time, uh about 15 years ago, that was did the same thing. And now they're all but non-existent, and that's Heroku. Uh now, I think Heroku's downfall I I think Heroku's downfall was twofold. One, Salesforce. Uh and two, um they had a relatively extreme opinion on how to do things. Um you know, Ruby first, that kind of thing. Does Supabase have those types of limitations, or are you more like here's a port, go forth and conquer? >> Yeah. We are not opinionated about how developers use our service. We support SDKs for various programming languages. So, clients can connect either directly through SQL through a SQL port or through HTTP or using these libraries as clients. So, we are certainly not opinionated about how developers use our service. It is in service of what they want to build. We are trying to enable people to build the things they want to build versus them how they should build them. >> Okay, so you're I mean, realistically, I mean, you're just basically infrastructure as a service. If you're building a data app, if you're building a database app, your infrastructure as a service. >> Yes, that's right. >> Okay. And so, the core services and and folks, understand and I don't mean any disrespect, I've never even I've never explored Superbase. I mean, I've looked at the website. I do know who you are, of course. Um, but I don't when I develop, I I fire up a VM and I do things locally or, you know, via GCE or something like that. That isn't to say that there isn't use here, it's just it's not my territory. What um, so I'm going to guess and correct me if I'm missing anything, you offer a REST API. >> Yes. >> Um, and that's going to be your HTTPS APIs, you know, REST. I assume you offer things like snapshot backups. Uh, those types of things. Do you offer point-in-time recovery at all? >> Yes, we do. >> Okay, so now that's that's cool, right? A lot of services, they're like your point-in-time is whenever the snapshot was. Um, whereas point in your true point-in-time recovery, I mean, you can recover down to the transaction ID, right? So, that's that's quite useful. Um, where are you at on your multi-gress? Where where do where cuz there's another company out there that is louder than you guys. And that's really only the only term I can use honestly. They're they're louder and that would be PlanetScale. I'm sure you know who they are. Um what's our what's our our our go-live date at least from an MVP perspective for the public for multi-gress? >> So uh we did an open source alpha release of multi-gress in June. And we will be making it available on the Superbase platform soon. Uh people will have to wait and watch for the announcement. >> Sure. >> But soon uh we will make it available on uh the platform for people to try out. >> So uh give us a rundown of what multi-gress is because you know we have Aurora, we've got PG Edge, um those types of things. So what is multi-gress? >> So the vision for multi-gress is that it's it's intended to be almost a new operating system for Postgres. Uh Postgres is the foundation of all of the services that we offer at Superbase and it's extremely popular among developers. Its popularity just keeps rising. It has a very active community that develops it. It is independently run. It is not owned by any big corporation. The big companies all run paid services that use either mostly forks of open source Postgres, but those forks [clears throat] are 98 or 99% uh open source code, right? So um the thing is that all the cloud providers who run Postgres actually build a lot around it in order to provide it as a service. People who are running Postgres on their own in their own installs also have to do that. It is not easy to administer Postgres. It's easy to get started. Just download it using your favorite download manager and run it locally. That's very easy. Whether you're doing it on Mac or Windows or Linux, it just works. >> Please God, don't do it on Windows. I digress. Please move forward. >> But but once you get to real production, it requires administration. There are gotchas. There are food guns. People have experienced them. And if you don't know what you're doing and you run your production on it, you can end up having a very bad day at some point. The intent and the vision for Multigress is that Multigress will manage all of that for Postgres. And on top of that, we will also do horizontal scaling through sharding. And sharding is the one thing that none of the existing providers of Postgres services offer. With with an exception, there's there's one exception. >> What's the exception? >> The exception is Citus, which was acquired by Microsoft and >> Right. Is it they call it Cosmos or something like that? But yeah. >> or Horizon DB. It's been rebranded. >> Right. >> So, that's the vision for Multigress that you can start with it. You will get HA out of the box. Now, in the Postgres ecosystem, there are tools that do many things one off, right? So, let's say you have a specific issue with Postgres, which is it's a single cluster. What happens if you have to upgrade it? There is no easy way to do it. There are tools that help you do this. If you want to run it in an HA configuration, highly available configuration, so that even if there's an unexpected error with your primary, you have something to fall back to, you run something else to manage that. So, for each problem that you are trying to solve with Postgres, there is a tool. But, it's up to the user to get all these tools to work together. And the vision for Multigress is you don't have to worry about getting these things to work together. Either we build things from scratch where there isn't a solution or a good enough solution in the existing ecosystem, or if there is a good enough solution, we integrate it in a transparent way, so that things just work. And that's just from the large Postgres cluster administration point of view. And then the next thing beyond that is sharding, which allows you to scale horizontally and scale beyond what a single Postgres cluster can support. >> Okay. So, one, I appreciate your vision. Um I've been doing this for three decades. Uh longer than the PostgreSQL project has actually existed. Um there is no database on the planet that can do what you have just said. Postgres or not. And so, I do admire the vision. Um but, you know, Amazon came out 15 years ago and said we're going to you will no longer need a DBA, which was a lie. If anything, they increased the need for DBAs. What they did is they removed the need for an SRE. Um you can remove certain complexities through uh template, through standardization. Um but you're talking and and I mean and you know this. This is why you weren't able to build it in a year. Right? It is a very you're talking about simplifying the most complicated program problem in modern databases. Right? Or I mean the first thing that was the biggest challenge was uh concurrency. True multi-user where you didn't block. Right? I mean multi-user's been around a long time, but in databases you had a scenario where you would block and cause another user to have to wait for you to be done. And after blocking I it became replication. Right? How to accurately replicate, specifically synchronously replicate, but still and have hot standbys and point-in-time recovery and things like that. Well, we solved all those most of those except for multi-master. Right? I mean multi-master's still a challenge in general, especially with an architecture like Postgres. You know, PGA just trying, but there's still a lot of limitations to their their architecture. Um they've more made more progress than anybody, but still. Um I I got a challenge you, you know, I I I get the dream, I get the passion, but I don't I don't think that you can honestly say we're going to remove or make it simple or it's just going to quote-unquote work. I just there regardless of Postgres, if you said it with MySQL, if you said it with, you know, Ingres, I I I don't really care. I mean, you've been in this industry as long as I have. There doesn't just work. There are people turning knobs in the back. Right? You may You may put up a nice piece of plastic up front and be like, "Sure, developer, it'll just work." But, you've got a lot of gremlins in the back doing a lot of work that they don't see. >> So So, the idea Well, uh let's let's go back first to multimaster. The contention is that you don't need multimaster if you have sharding. >> Well, I would agree with that. >> And uh as far as uh putting the facade that developers see and removing the need for developers to worry about these things, definitely that is part of the mission. But, part of the mission is also that uh it is in fact possible to to get a large portion of the way there. And it may take years. We are not saying that all of this will be done in 1 year or 2 years. Uh and the reason why I feel confident that it can be done is because it was done once before for MySQL with Vitess. So, we should be able to get to a point where yes, we still have SREs, we still have people who have to respond to alerts and do something on the back end. But, it's going to be uh a much smaller number of people that are required than the the DBA staff of quality >> Yeah, and and and and I would agree with you there. I mean, there's no doubt that as we I'll agree with you on two points. One, it's going to take years. Um and two, that yeah, it will be less people. It's already less, right? I mean, the the role of the DBA has been diminished and made more important, really, but diminished in terms of uh quantity uh for decades. Because developers, who are the people that business understands, right? If a business says, "Go do X, developer." Developer can do X and display what they did. DBA's not so much, right? You say to a DBA, "Go do X." And you talk to the business person, the person business person has no idea what the DBA did to get it done. Right? So, I understand that. So, if you can make your life simpler for the developer, you're going to see growth. Um you know, I I do wish you the absolute best with that. I think it's going to be more of a challenge than I mean, I don't think you're underestimating the challenge. I mean, you're obviously qualified, but I do think that the messaging is not as on point in terms of the level of challenge we're talking about. I mean, sure you did it with MySQL. MySQL's a completely different platform. There is no comparison MySQL and Postgres. Right? They do MySQL and I'm not knocking MySQL, but they're like I don't They're willing to put a you know, drop a firecracker inside of a a gasoline can. You know, Postgres just isn't going to do that. Right? They're going to say no and build a wall between the two. Um so, there's and you know, and there's advantages to both, right? One, you get a big explosion. It's a lot of fun to watch, but good lord, your data. Uh [clears throat] and and then the other one is is where's the fun? And the response is, "Well, the fun is in the correctness." No, it's not. Right? There's no fun in correctness. That's boring. But from a business perspective, it absolutely makes sense. You said soon. I know you can't give me a date. But can can this year? Maybe? >> Maybe. >> Kind of? Maybe? Okay. >> Maybe. >> May- maybe? Uh that's fair. That's fair. All right, is there anything else that you want to be able to share, to explore with this cuz I mean it's your sh- it's your mic. >> Okay. Um So, when we Well, I do want people to watch out for the announcement when it becomes available to uh to use and we would love for people to try it out. And initially, it is going to be free to try out. So, people will be able to try it at no cost on the Supabase platform when it when it is opened up to uh the the full full community of developers who can possibly use it. Uh so, that's some that's a thought that I want to leave our audience with. >> Okay. Well, um everyone, let us uh make sure we watch for that. Uh Supabase has been very kind to Postgres Conference / Postgres World as well as my own company and we don't even use them. Uh so, that's uh you know, that behavior, that reputation stands on its own and I really recommend that people give them a shot. Especially from a developer experience perspective. If you are a you know, random developer who's halfway gifted that really that wants a Postgres back end and wants no trouble, uh Supabase is where it's at right now. I mean, RDS is great, but RDS has a huge infrastructure, uh you know, an organizational bureaucracy around it to use it. Supabase, you go, you sign up, they give you your API key or your your credentials and you're off and running. So, everyone, give Supabase a a shot and Deep D, thank you for joining us. >> Thank you for having me.