Databases were not designed for this

Okay, yeah, long day. Awesome, final talk of the day very likely. What I'll be talking about is what databases were not like the basically the databases were not designed for this and here that this means agent tech world that we are all living in. So yeah, brief intro of me. My name is Arpit. I am a principal engineer to at Razorpay. Write a lot of stuff on socials and my website and blogs and all. But at Razorpay take care of three things. It is just been two months like very young in my Razorpay journey. I am leading agent studio where we plan to provide harness for merchants to build custom agents. That's one product. Second is I look at merchant developer experience which is including docs, CLI, MCPs, SDKs. And third very recently, literally yesterday, I started looking into customer data platform which is essentially Razorpay has merchants, merchants has customers, that customer has lots of insights and we try to crunch insights out of that. Right? Awesome. So, what we're going to talk about I'll go to full full screen full screen. Sorry, forgot about it. Okay. What we're going to talk about is the existing database architecture that we have is something like it there were a lot of implicit assumptions that we made with that. But now with AI kicking in, they are becoming more explicit. They'll be like, but who lets agent talk to database? Who's that stupid guy? Yeah, we all are stupid. Not directly. Basically not letting agents draft SQL queries and fire onto the database, but even through API that we do, right? Through MCP servers that we do. I'll give a very concrete example to make it very clear. So for example, world is heading towards building a natural language to query interface, right? In which you would want something like a self-serve analytics where people would say, "Hey, what is a like let's say a CEO one day wakes up and says, "Hey, I want to check the trend line for last 5 years of revenue that's let's say coming in." Right? Now, it's a very natural language query that's there. Now, this query this this natural language input gets converted into query which fetches the data, serves it. Right? So, it's not just that why would agent directly query the database. It may not be like direct query query getting fired, but where the world is heading, we are certainly seeing a place where a natural language input gets converted into query and that gets fired. Now, what I'll walk through is four assumptions and fixes for each one of them. So, assumption number one that the world was living with was deterministic caller. Now, deterministic caller is we exactly knew like imagine pre-LLM time. We exactly knew what the queries were. We exactly knew how the data was laid out. We decided indexes as per that. We decided our database caching layer as per that. Our query planner leveraged the statistics at a table level at an index level to formulate the best query and fire. Right? But, what changed? What changed is that given now the queries eventually will be very natural languagey query, your agent might join five tables that have never been joined before. You never anticipated that these five tables would ever be joined. Or, your indexes that we typically create covers a happy path. But, now you're leaving it to our agents to craft queries given any natural language input, craft queries. Now, they have a free hand. Whatever schema they feel like, whatever columns Oh, sorry. Whatever tables they feel like querying, they'll join those tables, query it, project star, project subset of columns, whatever. And that would take your database for a spin. Problem. So, fixes. Very simple things All of this already exist in the database. I'm just trying to uncover them. They were all buried. We all never worried about it, but now we have to. So, first thing, statement timeouts at a role level. We all know all databases supports our back, but we never bothered creating more than one user. If you are building your own Instagram, you would create database called Instagram and user called prod. Everybody did that, right? On a good day where your prod becomes a staging, you create another user called staging so that like for some reason you do that. Most people just happy testing with prod user. Okay, that's how startups work. So, the time has come where we create roles, the R backs of the database that we already have leveraging that. Two key parameters to set is statement timeout, which is how long one statement would take, and more importantly, idle in transaction timeout. So, when you are in transaction, what is an idle time after which you should like in within which if it does not wrap up, kill it. Because you do not want your connection to be held or a transaction to be held for longer than what you need it to be. Next up, soft deletes. We all know soft deletes. We all know why I'm how important they are, but with agents, it's important to know deleted by and delete reason. Every agent is doing a multi-step reasoning. When it decides to delete something, there is a reason for it. Capture that reason and put it in the database. You like, "Arpit, but Arpit agent is not firing delete statement, right? Arpit, but MCB is doing now. Even you have an API, your agent is calling an API to do it, pass the reason. The core reason behind passing the reason is to know the reason why that why the delete has happened, unfortunately, right? And this also helps you debug stuff. Like I was prototyping something and agent deleted my data. Like that happened with everyone, of course. It deleted a row that it should not have deleted. I was building a small application for my daughter to do something. And uh what happened was it deleted a certain set of rows which it should not have. And uh after that I added this column. Then I let Claude do something. It deleted again, but this time I had the reason. Then I did it soft deletes and I knew the reason and that was how I uncovered a bug in my agentic basically multi-agent reasoning step that uh my prompt was having a issue. So then I explicitly said, "Important star star, don't do this." That's all we are doing like it's all English now. Next up, append-only event logs. So, for some like high-stake rights, let's say you're doing an update. Let's say it's a not even like not just financial systems in general, but any rights that you're trying to do again in our agentic workflow, it is going to manipulate a set of rows onto the database. In that case, make sure like not make sure not always like you would want to do in-place updates by firing update query, but there are certain high-stake rights where it's better that you make your right as an event and then replay the events after a human intervention or a human in the loop because there are some high-stake rights involved. And I got you would you would be very cautious by doing such rights, but just in case you're doing rights with agent or it or modifying via API, MCP, whatever. If it's very high-stake rights, you could basically take you so you could typically make your database into an append-only event and then decide how you'd want to apply the changes after a review and whatever, right? Here you would see an important string which is item potency key. Next up, that item potency key were optional. We never bothered about it unless we knew retries were going to happen. But most people were happy assuming there would be no retries. But with agents, retries are inevitable. Cloud will keep retrying. Oh, I tried this call, this call failed, I'll retry again. But in that case, imagine you are just bluntly appending, appending, appending to a table or inserting, inserting, inserting to a table. Now, the problem is like, "Arpit, I have unique key." But if you have unique key, then you would run into runtime issues. It will fail when you're trying to insert, right? So, that's why you have to check and set. But unique key, if it is auto-incrementing, then that's why you need to have an item potency key. That for this operation, now what operation? For example, if your agent multi-step, whatever, there you have this is a task. For this task, for this step, if this is my plan, my entire plan's hash can go in, and wherever it's applying, like whatever entity it's applying, right? You could again, this is a very trivial example of an item potency key, but it's very use case specific on how you'd want to build your item potency key, right? And just use that item potency key, and pass it, and store it, so that even if your agents retry, now retry might not just happen uh during one workflow. You also have checkpoint and resume, where your agents died due to whatever reason, and it resumes. It would start from the top. So, you do not want, although you would let your you would let your token usage go up a bit, but you certainly do not want your database to go into an inconsistent state if you're choosing to apply updates using tokens uh using LLMs. You'll be like, "But are you ever going to do this?" Uh it has been good 2 months. I'm running my Gemini on yellow mode, and Claude Claude on bypass permissions say skip. Sorry, get a little busy, dangerously skip permissions mode, right? Now, I don't even bother. I know it's not going to delete stuff. If it Again, not in prod, of course, but yeah. We are going We're inching in the direction, so make systems foolproof. Next up is kind of pull request for a database. Again, going into high-stake rights a bit more, where we were where we discussed like uh where we discussed this up-and-only event logs. But, if there are even higher-stake rights, where your event log could be human in the loop, this is very similar to that, where you might want to have This is not like applying events, but this is you'd want to apply the rights to the database, but after an approval from someone. And like until then, everything stops, right? This is something that comes in very handy if you're building, again, a multi-agent workflow with human in the loop, where an approval is required before it pushes. For example, someone trying to create a stat report to send to an to, let's say, investors, and you'd want to just double-check your number before stuff happens, right? So, similar to that, if you're taking any actions, this becomes your approval queue. You go through that, and someone approves it, "Yeah, this looks fine. Push it, right?" That's next. Okay. Assumption number two, updated shell, which is connections are brief. We never like We are all users of OLTP systems, like MySQL, Postgres, Redis, SQL keys of the world, and what we almost always see is, "Hey, it will reply in what? It will respond in what? 3 milliseconds, 5 milliseconds, 15 milliseconds, 25 milliseconds." We almost always assume that the connection that is being held is just held for like less than a second, But, that's not true, because agents think, or LLM takes time thinking. That's problem number one. Problem number two is they fan out. So, one agent might spin up five sub-agents to do things. So, your number of each agent, let's say, wants to talk to database, problem. They're all establishing connections to it, and issues. Then, agents also multiply quickly, as if they're reproducing, but yeah. Three agents in dev become 30 because you got a surge in traffic. So, the problem is if each of your sub-agent is going to create connections to your database, your database will be overwhelmed. And which is what we have to worry about. So, fix number one for that. Have a dedicated connection pool for your agent. You would of course you're not like although agents are doing a lot of stuff, but they're not replacing your transactional use case. Where your user is interacting with your website or app or whatever, right? So, you would have ideal thing is have separate connection pools, one for your transactional API synchronous use case, and second one for your agents to consume. Because agents are okay being slow. That's one good thing. They're not >> [laughter] >> They're not like humans who want everything fast and let that classic Amazon study which says, "Hey, we saw 5% drop if my page time load decrease or it increase by 100 ms." Like they're not They're not that like agents are not that stupid. Humans are. We want things to be fast. So, given that agents can take their own sweet time and they're okay taking longer time or if our downstream systems take longer, leverage that and hence have their own connection pool. They're okay waiting, not end of the world. Next up. As By the way, if you observe here also I have the same two statement timeout and in another transaction timeout added part of my code. Okay. Next up. Never hold like your transaction during thinking. This is a classic mistake that we all used to do with you have a database transaction and within the transaction you're pushing message to Kafka or making a network call and you say, "Oh, my transactions are taking like like Oh, sorry, my overall my connections are being held for a long time." The fix used to always be that, "Hey, you keep your transaction time as short as possible and offload it to a thread which then writes to Kafka, right?" Same concept, but here it's not just write to Kafka or network call. Here it's literally an LLM invocation that you are doing. This one. Where you would want your LLM to say, "Hey, should we do this?" This would take its own good 20 worst case like not worst case but average case is like 5 to 7 seconds easily it would take. Why do you want to hold your transaction for that long? Keep your LLM thing above once you have all the data for you to perform the transaction, then you do it. Like try to do it again. There are some cases where it is unavoidable to do that where you want the transaction start and after that depending on a branching condition you would make a call to LLM. Right? That's possible but try to minimize such use cases. If you understand your use case well and if it can be structured such that all LLM thinking can be brought on the left side or at the top and then you perform your transaction. That's always a good practice. Assumption number three, two more to one four assumptions after this one and then we are done. Next up, assumption three, bad queries fail loudly. You always assume this. Like again, it feels weird to say it out loud but we almost always assume where your bad queries were like in a human operated world, it was like if there is a slow query, my dashboard dashboard is sluggish, you get customer tickets. Hey, your dashboard is more slow, not working, right? Or if your queries were slow or bad or had errors, it used to bump up your 5XX. And then you used to get pages, right? So, they used to fail loudly. But this changes with agent. Why? Number one, agent is okay when things are slow. So, again, if there is slow result, agent will wait. It's very patient. Humans are impatient. Agents are patient. So, even if your results are slow coming from your downstream systems or sorry, coming from upstream systems, they are okay. More importantly, agents are also okay if your result sets are empty. Now, think of it. Imagine you're building a search system. A normal filter-based search, and whatever parameter you pass gets gets compiled into a SQL query, fires onto the database, gets the response. Now, where you pass in a typo or something which is something that doesn't exist in the database, you don't know if it was a wrong filter or if it was a correct filter, but there was no data matching it. Now, the problem is agent would see no results. Agent will operate with that. So, the risk here is very high. The query was bad. What it compiled was bad. What agent did was hallucination. But, what we did or rather we just executed the query, got the response, no error, no rows found. Agent started working with that. So, hence adding that additional check, if you're having similar use case, having that additional check around the parameters that you're passing, is it legit legit? That's why enums agents love LLMs love enums. So, use that very well. That so that you're not digressing much. And hence, your no result found or your query or your data does not exist or your no result found is like like makes sense. Like you're basically differentiating between the two because agents can't. Next up is how to fix it. This is very simple. We Again, most never did this, but now very important. Where given agents are firing We're given that agents are firing queries left, right, and center, what would happen is um we always monitor slow query logs now. But, agents are okay with slow query logs. But, we need to know which agent fired which query. Again, via API, via MCP, via whatever. If it's originating from agent, log it. Now, how? SQL statement has comments. If anybody has you If anyone among you has used ProxySQL or any proxy, you know what I'm talking about, where you could have a regex-based routing at your proxy level. Let's say ProxySQL you're using, which is abstracting out your database topology. You used to add, "If I see this pattern, route it to this node. If I see this pattern, route it to this node." Similar to that, you add comments to your SQL query which says that, "Hey, this query originated from this agent." From this agent, for this task, on this step. Simple, right? You tag as much as you like, and then you can filter. For example, all these queries are eventually flowing into your logging system. Now, in that, you can apply a simple string-based filter to filter out queries fired by a particular agent. This helps you debug. You don't have to explicitly log these queries someplace else. You can You can use your existing infrastructure to debug slow queries, which queries were fired by which agent, so that you know the trace. You know who did what. Now, you know, because of this, you would know, "Okay, oh, this agent is creating this query almost wrong every single time. Let me fix it." Then you update your prompt, things work fine, you add evals to it so that it always almost always generates the query that you're expecting it to generate, right? So, very All these tools, the good part is all these tools already exist. It's just that we never bothered to use it. Agents are making it apparent. Next up, active circuit breakers. So, here, if let's say one particular agent, kind of continuing with the last example itself, if one of the if one of the agent that we have is consistently firing query which is leading to syntax error, you can you can create a circuit breaker for it. For example, have something a super simple query like this that's checking group by agent ID and tell me which agent is leading to most failed SQL queries that are executing on the system or queries that are taking the longest. You log it, you put a alert, you put a threshold, and chop that agent off. Because if you don't, then the agent will chop your database off. One of the two will happen, right? So, it's important to have this again, all of these tools, we'll say it again, already exist. We're just not using it, right? So, the whole idea is if you observe the central theme that's emerging is to have very high observability built on top of your database. Although I'm taking SQL as an example, holds true for query tagging that you typically do in almost every database for you to know how that like how is that database being used, etc. etc. So, we're just piggybacking on this existing stuff itself. We're just surfacing those things out explicitly. Next up, last assumption, which is schema is for engineering. So, we all have been there, done that, creating column names that look short and cute like USRID, statCD, FLG1. If you have directly dealt with your data warehouse, you know what I'm talking about where column names are almost always underscore one underscore two because you don't delete columns because if they're expensive and then you have problems with that. Right? So, schema is for engineering was an assumption. Uh it was being used by engineers who has a lot of context on the system as assumption. If not engineers, analysts has context on the system. That is also an assumption that if you cannot figure out what is this data for or what is this column for or how do you crunch this data? You typically go to your analyst and say, "Hey, I want to crunch this data. Tell me what to use." And then the analyst, he or she comes up with an absurd-looking query which you make like who names column like this? So, whoever does it, that's a problem. But now that given that agents for a self-serve use case where natural language to SQL to execution part, if your column names are not verbose, your agent will be confused. Your agent will not understand what FLG1 is. Oh, you can add column metadata, but it's better to give a better column name. So, instead of using calling, let's say user ID, use customer_id, not cust_id, but full customer_id. More importantly, look at the fulfillment status. It's like legit enum at my database level. So that acci- even accidentally no random input is being passed to it where agent hallucinated a status, let's say NA or unavailable, which does not exist. Right? You don't want that to ever happen because agent can come up with that status. With this enum in place, it would get either an error when you fire the query que hey, this does not exist, or you could apply this check, bring this up at your metadata level, at your data hub or whatever you are using to show que hey, these are my available options, and when you craft a query, adhere to this. Right? If you observe, we are just taming the lion here. That your lion can go anywhere, you're just putting a ring around it. So, he just jumps from one ring to another, and we make money. Right? Next up, scoping the blast radius. Now, again, same stuff that we started with, coming full circle, is uh we typically, or rather uh true for FinTechs more than others, where a lot of tables and columns have this security baked in that only certain set of users are allowed to access the table. Even production users, production database users are not allowed to query. Same thing. So, if you, as an agent, you're firing the query, let's say a particular agent, let's say uh fulfill I'll just take the same example. If you let's say I have a I have a fulfillment agent. So, fulfillment agent should be able to access like grant select and update only on fulfillment status, ship that, and tracking number. And not anything else. Because the moment I open up everything to that agent, agent sees full schema, will go full berserk. Like yeah, this is the ideal state of my data. You don't want that to happen. So, again, taming the lion. So, this way you put what it can select, how and what it can update, what it can insert, which column it can access, very important. Seguing into the Postgres is world, where you have row-level security, this is additional level of security that makes life even simpler, where you don't have to like Okay, for context, what is row-level security? At low at At row-level security, you could pass a setting, something like Hey, where did my mouse go? Here. Like app.current task ID. This is a very simple example, app.current task ID. Which is like, "Hey, I am belo- I belong to this task, and if I've added that row-level security, on which rows am I allowed to access?" Even I Even if I fire select star on the table, it would not give me other rows. It would only give me rows which I am allowed to access. So, this is this is like literally row-level security. You fire regular Even if your agent fires regular select star, it would not get rows that it does not have access to. And this is how, by the way, your serverless Postgres offerings work. If you have gone through Supabase's stuff, it is filled with row-level security. If you have gone If you used Firebase, it also has row-level Of course, it's not Postgres based, but it does also row-level security, where you're defining what rule you have, and what it can access. So, you don't have to have this task ID that you're passing over here as a where clause. You just file a regular select star and automatically filters out stuff for you basis the rules that you have configured. Right? That's very good hack. Summarizing, and by the way, that's the end. Summarizing few things that we discussed is you are building your data layer to be defensive. It's better to be safe than sorry, especially with databases, because once your database goes into an inconsistent state, difficult to get back on track. So, always go with list privileges, which is have row level timeouts, per agent roles, have traceability in place where we discuss soft deletes, up and only logs, query tagging, even approval queue. Then have resiliency where we discuss item potency and high stake approval queues as I mentioned. Then isolation where we have row level security for Postgres base other data I don't know my sequel added. I don't know I don't think it's there in my sequel yet. And then namespace isolation, of course. Then safety and legibility. Again, safety is about circuit breaker and transactionless thinking. Where you are not By the way, what transactionless thinking is that your your connections are not brief, your LLM call should not be within transaction, that part of it. Right? So, the whole idea again, summarizing stuff, is that your agents don't break your database. They just expose assumptions you didn't know you had. And yeah, that's the talk. You can find me on Twitter, YouTube, LinkedIn, everywhere. I'm perpetually online. But yeah, that's my talk. Any questions? >> [applause] >> No questions? I'm always happy to make no questions. Either it went really well I don't know. No, I'm pretty sure it went very well. >> [laughter] >> Okay, thank you so much Arpit. If there are any questions, I believe you'll they'll be able to reach out to you as well. >> always. Please. >> Um so, when agents are updating the database schemas, >> No, no. Don't let agents update your database schema. Schema name wrote. It's okay. Why would you want agents to update your schema again? They would out because you would anyway review your schema, right? Yeah, if you can explain like if you can elaborate on one use case, I'll give a concrete answer to that. Huh, so that's a toy project. So that's like your zero to zero point one journey. In that it's okay. But once you roll out in production, after that you would not let your agent update your schema. Rows are okay. Schemas are not. Schemas because you still have to take a look, right? Like what are you doing? What column? Like if you let your agent if you let your cloud design a schema, it will go full Google L6 level schema. Huh. If I'm thinking I like I would think on the fly the way I do it is but again I would still like my first inclination is to say no to this. Like never do that. If you still want to do it then in your prompt or skill or whatever however your agent is creating it using agent SDK or whatever, you have to have those guardrails like how descriptive your name should be. If it needs to be in 3NF form, have a critical refiner loop. Like have a multi-agent loop in that which says that hey this is my schema, critique it. If it once it critiqued it, refine it, right? And you iterate until you have some sort of way to know what score it is. I was working on something very similar to that but where I was building a small agent that helped me review code. And while reviewing code Go guy wrote but then the schema came in. So whatever I do as a P key okay this is how the schema should look I was trying to just document it. So I use it for review I did did not but I now that I think of it I could use it as a as a guidelines to write schema but like that's for me is like a refiner loop that I'm putting in but I would still say don't do that again thanks for adding that example lovable doing that stupid stuff. Yeah yeah. Mhm. Uh-huh. Yeah. Mhm. [snorts] Mhm. Yes. Yes. Yes. Yeah. After schema I could still do it but schema pay trust your instinct more. Yeah. Okay. Any other all good? Okay okay. Huh that's what I said like not directly firing query but at least we are MCP it would do it. Imagine self serve analytics as a as a concrete example. Right? They would still because from a natural language it has to create a sequel. Because and you need that natural language interface because you want everybody to all to have like democratic access to data. You don't want it to be guard railed or you don't want it to be gate kept by your analyst team because then they become the bottleneck. So, we're inching towards the fact that eliminate all sorts of bottlenecks that you have in your org. Right, so one at a time. So, this is one of the one where you don't want to have so many analysts who is like gate keeping all the stuff for you. Plug-in to do what? All of this already exists, right? So, I've not done any of that. I've used existing features to make it more robust and that's where I stopped because it had already it had it already had the things that I wanted. Max connection per role, yes. Yeah. Plus if you put conductor, so buzz goes berserk on that. Got to nice. So, awesome. Thanks a ton, folks. Yeah, another round of applause. Thank you so much for that amazing talk. Yeah.