MCP at Scale: Governing AI Agent APIs When Autonomous Systems Are Your Consume… | apidays India 2026

Thanks. Hey everyone, good morning. Uh [snorts] hope everyone attended the uh keynote session just before the session. Right. So this session is going to be aligned with that and we will get more into how to uh securely and safely use MCPS uh when your agent needs uh more and more number of uh MCPS at scale. So that's about the session. Hope I'm audible, right? Okay. Uh okay. So with AI agents becoming more and more popular, right? So the number of security breaches and vulnerabilities that happens is also becoming very uh high, right? So recently in July, there was a security breach that happened in hugging phase. Hope you might have uh seen about it, right? So how do we set all the proper guardrails and governance? So this session is going to be more about that. uh how to uh govern the MCPS that are connected to your agents and what are the guardrails that you need and how the API management is now translating into AI uh gateway management. Um so that that'll be the session today. Um so to to uh before we get into the session to give a a brief intro right so uh what is an agent? So uh an LLM model uh like claude or open a uh assis is not a model right. So an a agent is something a model plus a harness around it. So for for a [clears throat] model to function properly it needs the proper context. It needs the proper functions the tool call the harness the governance. So everything together it forms the uh AI agent. So model is like the brain and harness is rest of the body. So that's how uh uh they determine it. And how do the harness gets all what it is required? Context is a subset of harness. The governance is a subset of the harness. So how do uh uh we get the right harness uh that has been passed on to the uh models. So probably most of the time it is using MCPS. So if you are working with any coding agents right so hope you might be using co-pilots and curses. How many of you using copilot and curses? Okay very few. Okay so you might have configured MCPS to get your Jira stories to get your code from GitHub to create a pull request. So nowadays uh I see in my team people doesn't uh run any commands to start a server. They go to copilot and they give a command to start the server. So things have been changed the way how we are working right. [snorts] Uh so this is about uh a quick brief about uh what an AI agent is. Uh and this is the evolution right? So it started with rest APIs when rest APIs came in. Uh so I used to work with SOAP and then when REST APIs came in there is a lot of uh popularity around API gateways to make uh a rest API more secure uh and it had governance around the uh rest endpoints and still it is happening until today. uh and then when microservices came in when a product or a platform is built with multiple micros service uh there used to be something called a service meshes it came up with its own set of capabilities and when uh mobile apps came into [clears throat] uh the market right so since uh 2013 there was a more uh uh traction on the mobile apps now everyone use a lot of mobile apps right so then uh the concept of BFF came in BFF and GraphQL back end for front end and the GraphQL uh came into that and now with AI agents. So the AI gateway is something uh the AI agents uh definitely need to get the productiv productivity gain that you're looking for and to make your agent execution more safer and secure at the enterprise scale. Okay. And there are six levels of uh agent autonomy. Uh again if you have noticed in the keynote session today uh we were somewhere uh in between a chat assistant and an autonomous coding agent. There are around uh six levels of agent autonomy. It started with uh an assistant. Uh if uh if uh you were using GitHub copilot during the covid times, it is mostly an autocomplete way. So you give in the method signature and then the GitHub copilot would fill in the method details for you. And then we went into a direction after chart GPT came in where it was a supervised action. So we give the query and then the agent gives the entire response in the form of code. If I need to add two numbers, I just give that as a prompt. uh I get the function whichever language it is and then we entered into uh the co-pilots that we are using now the co-pilots and cursors that we are using now uh where uh we give the entire codebase and it runs in an autonomous mode but still it requires a human intervention to do uh a turnbyturn mode and then there is a level called level three which is goal driven. So you give a goal uh to the agent uh and it needs to complete that goal for you. It could be to write a complete feature from Jira or it could be to do a code review. So you give a goal and then it works on that goal and then there is something called the parallel delegation where multiple agents would be running in parallel and this is how uh the software engineering is trending towards right. So uh they are envisioning software engineers are going to be the agent managers where they will be managing fleet of agents. Correct. Uh so the parallel execution is where you give a goal and then there will be multiple agents uh which will be working together to complete the goal. It could be starting from uh doing a discovery of the requirements and then to write the code to do the testing uh to do an automation testing run a sonar or fix the sonar issues and then at last merge the code and get into a deployment. This cannot be run with one single agent. We need multiple specialized agents. Right? So it starts with uh the requirement agent and we need a deployment agent. So that's where the parallel delegation uh comes in. And then the last one is managed by exceptions. So this is a fully autonomous mode wherein things operate autonomously. It doesn't even need a human agent manager rather based on the events the set of agent ecosystem will trigger and it will achieve the goal based on the event. Uh so until L2 which is like a scoped task where the human would always be in the loop. they will be doing a drive by uh drive turn to achieve what they need through the agent things are fine but when we get into the L4 which is uh the goal driven uh and then L5 and L6 things become more autonomous the level of human intervention uh in these autonomous agents or human in the loop in these autonomous agents would be very minimal and this becomes a uh this creates a very high risk right so things operate autonomously you doesn't know what happens uh it it is going to be like a black box but then things should happen very safe and secure. Your goal should not create an organizational level breaches. Okay. So to give you a brief example, right? So your uh a year back we were working on building a super agent platform. Okay. So where the platform would consist of an AI PDLC, the AI SDLC and AI ops. Okay, you could relate that right? PDLC is for product uh development, SDLC for software development and then ops is for the post deployment purpose. So we were building a super agent platform which composed of at least uh to start with 30 agents and there were 100 plus MCPS. So if each of the MCP is exposing at least five tool calls or five methods then there will be around 500 tool calls that can be associated with this agent. Okay. So each of this agent would get access to all these 500 tool calls. So now you can envision the scale. So this is where we started with. Now we are completely in a different scale right and this super agent platform uh would operate on these MCPS in an autonomous way. So now then came a problem to us where during the development phase one of the MCP that we used obviously we were not that experienced right we started pulling some MCP servers from GitHub and we configured that directly with the agent uh we noticed some kind of a uh issue wherein uh the MCP tools started deleting some of the files after uh a goal is driven or or after the tool call is done or after the goal is achieved. So that's when we noticed governance is an integral part of any agentic platform. So we can't give an excuse to not have the governance. Okay. So that's when we started how we can put all the guardrails and governance so that instead of configuring MCPS to each of the agent how can we centrally do that so that all the 20 agents or any new number of agents that are onboarded into this agentic platform there will always be only one MCP and that would make a federated call to the list of other MCPS uh whenever needed by this any MCP can be easily plugged in and it can be used to the agentic platform without any code change at the same time a single MCP which is going to federate we will be enabling all the guardrails and governance and this we started uh calling it as MCP gateway and now today we are evolving that as an AI gateway okay um so probably we could have gone with API gateways so most of the things are AP calls right Jira exposes API GitHub exposes API any tool that has been in the industry for more than 5 years are already exposing APIs we could use that why are we not using it and why do we want MCB gateways right uh so again I will quote a reference so as in the keynote session the API gateways are not designed for AI agents or for MCPS so the purpose of API gateway is completely different so predominantly the API gateways would be used to do rate limiting right so when there is a a notorious number of hits happening to your API it will do an API rate limiting and it does an health checks right so readiness probes and liveless probe whether your API is up or down to ensure your golden signal it can do that but with MCP gateways I don't need any rate limiting or with health checks so that comes in a different way correct and latency monitoring is a key aspect of APA gateway but here I don't do anything with uh latency monitoring so I need a output for my goal I don't care about uh the latency part and then with API gateways it is a static documentation so they write open API specs or swaggers it is exposed uh as a swagger doc or Open API doc and that is the documentation which is very good for human beings but would it help for AI? AI cannot even read that swagger documentation. So it needs a different way how it can read uh the document and then it can use the appropriate tools. Okay. And then AP works predominantly with API keys right. So you register your API into any of the API gateway. You use a client ID and secret get a beer token or API key and then you can get access to the uh APIs. And most of the times what I have seen is the validation happens on the uh origin or on the spring boot microser or whatever microser it is right it works based on the API keys but here with uh agents I don't want to give an API key and open access for all my tool calls and functions. So it should be based on the agent identity. We will come to the agent identity part. So it uh it it depends on the agent identity and depends on the user who is using that agent. Okay. Uh and then uh there is going to be a per request billing for API but here we all know it is based on the tokens. So these are few things why an uh API gateway is not sufficient to build an agentic platform at scale where it is going to be lot of MCPS associated with that agents. Okay. And uh so this slide explains about like what happens without an agent management layer as I said uh whatever we faced while we build that super agent platform. Right. So uh with an API gateway or even without an AI gateway. So these are few things that would be become a problem when we scale. So it could be in the form of data excfiltration. So there could be PII data. There could be some sensitive data. So without having a proper gateway so you are exposing all your sensitive uh information to those MCPS. You may or uh you cannot trust that MCPS but then it flows into that MCP servers and then uh uh runaway cost right. So you won't get the budget. So you cannot attribute budget to each of the calls. So whatever the AI uh needs it will continuously do iterations it'll use and then your token cost is going to be exorbitant and there'll be uh overboard access as I said if I am not giving any um security controls so I'll be opening access to every tool that I have in my orc and then the agent is freely able to access anything and the user may not have the proper hour back but still he may get uh all the other information which is not intended for that user right [snorts] uh and then the quality DK right with APA gate always it is always 200 okay but how would we ensure the quality for uh AI agents it it it is not with uh the HTTP status we don't deal with HTTP status right it is based on the eval scores so whether the AI is hallucinating or whether uh it is having the proper confidence it is having a completeness we can use LLMs as a judge so all that comes in the quality DK and that will be embedded as part of your MCP gateways and cascading failures right so u wherein in API gateway. So when there is too many requests to your API, so you will start throwing 429. But if I throw an error uh in an agentic platform that I explained the pipeline will broke. Okay. So then in that case I would need uh a a better mechanism wherein I need to have a fallback control. So I might be using GPD 5.4 but if that model is not available then I should be able to switch to a secondary model or a tertiary model. [snorts] either I need to gracefully handle that situation or I need to give the uh response to the goal that has been asked for. I cannot break the agentic pipeline in the middle. Okay. And then uh if you are not having a proper governance control, you will lose the audit trial. So after a month if you want to uh debug a code which has been uh generated by an AI agent which is producing some production issues and if you want to trace it, if you not have a proper audit trail, you will not be able to trace it. Okay. uh and it is not just for coding agent right even if you're using any agents for your business uh there are a lot of news uh uh of enterprises using AI agents and then the agents are not responding properly one interesting thing is uh Chevrolet introduced AI agent in their dealer showroom and when one of the dealer came sorry one of the customer came in and they were inquiring about a vehicle the Chevrolet agent gave a response recommending a Tesla vehicle okay uh so if I want to debug why it happens so I need to have that audit trail or else all these are nondeterministic uh until I know what the tool call it is made the timestamps what is the request what is the response I can't even debug that so we need a proper audit trail so these are few things that we will miss if we don't have AI gateways or MCP gateways uh to scale the agentic platform and again this is a quick comparison so the consumers for API gateway are human developers but for the AI gateway or MCP gateway it is going to be agents uh the contracts there it this OpenAI spec here it will be the typed tool schemas. So how many of you have written MCP tools? Wow that's a quite a big number. Okay so then you might be knowing it right. So when you define an MCP tool it has a uh description a tool name and then a typed schema. So those are the contracts for AI agents to identify and determine whether it needs to make this tool call or not. It is not uh in API gateway it is the open AI spec. So that wouldn't work here. Uh and then we talked about authentication and the capacity part. Uh and the governance part for API gateway is just a call but for MCP gateway or AI gateway it is going to be a decision. The decision can be in the tool call or it can be on the model whether the primary model or the secondary model and the quality. So based on the eval score it would determine whether I need to continue or I need to retry. So all that comes into the AI gateway and there would be some additional extra layers which are not part of the API gateway are like the guardrails and uh uh you put the uh governance part in it. So which doesn't come with the API gateway. So this is the highle architecture of uh the MCP gateway. So wherever you see MCP gateway or whenever I say MCP gateway just replace that with AI gateway. So this uh AI space is growing very fast. Right? When I was preparing this slide it was MCP gateway. Now AI gateway is becoming more popular and MCP gateway is a subset of AI gateway. So replace that with AI. Okay. Um so this is in a high level architecture right? So the consumers are AI agent and then you have multiple tools. So here for easy reference I have put it as uh uh Jira uh GitHub and code repo and other LLM providers as well. Right? So it could be even your APIs if you have any legacy API and if you want to expose that to agent you could still uh do that very well by wrapping it with an MCP. So if the AI gateway layer is not there, if the middle layer is not there, what would happen is the AI agent would be configured with all these MCP calls and it is free to use everything. But then the MCP gateway plays a role in between the AI agent and the uh MCP tools and that that is a vertical where MCP gateway plays. So every tool call that an agent makes the data would be inspected the data would be analyzed authenticated and only if it is valid and only if it follows all the governance and policies it will be allowed to make the tool call. If not it will be blocked there. So you will be defining n number of policies as you need into this MCP gateway and only if that is qualified it'll move to the tool call. If not it'll be blocked then and there and this policy will be applied both during the input as well as during the output. So you can define a policy to apply only during input or apply during only output or both. Okay. And you know the reason also okay so there are seven disciplines within the MCP gateway that we have to rethink uh uh when while we are migrating from API gateway to these AI gateways for agents right. Uh so the disciplines are like the transport uh so wherein most of the times in the API gateway it is predominantly the health checks but in MCP gateway it is not the health check rather it is the federation of the tool call. So as I said your AI agent will see only one tool call. Okay. But then internally it will federate to up to 500 or thousand tool calls. Okay. So the federation is a key part for these MCP gateways. It could be an H stdio based federation or it could be an HTTP based federation or even it could be a command line based federation. The next one is uh access. So uh rather than a static API key so you could make runtime tool call approvals. So based on the agent identity and based on the user who is using your system uh maybe you might have a web application in which an agent would be configured. So based on the user login and the arbback and based on the agent identity you will have a dynamic uh approval system and based on the agent identity there will be a set of scopes that will be assigned to the agent. Uh so based on that a policies would be applied. So we will see that in detail in the subsequent slide and then the contracts. So uh how to enforce a guardrail. Okay. The guardrail can be the rate limit or the guardrail could be uh anything like redacting your PIA information or if there are any sensitive information you should block the request. So all that would become part of the uh gateway contract flow profile. So this is the critical part. Okay. So I have an access control I have the set of guardrails. I have a set of policies. Now I need to package it in a way and then assign it to each of the request. And that's where the flow profile come in. So probably uh what will happen or what we are doing in our super agentic platform is so we we have any number of customers as I said we have a PDLC SDLC and ops right? So then you could think there'll be product users there'll be engineering users and there'll be an S sur user okay I'm simplifying this but it is more uh complex than I'm just explaining but at high level there are three different set of users and they'll be using their own set of agents. So now I need to create a separate flow profile. There could be a profile for product team where the profile would have policies and guardrails related to the way how they invoke Jira, the way how they invoke to any analytics tool, the way how they invoke to any journey tools. But when it comes to the engineer engineering uh uh SDLC part, the flow profile will be the rate limit on the number of API calls, the number of tool calls to GitHub, the number of uh calls to write the code. Uh the rate limiting uh so if a engineer does a very huge coding task, there should be some rate limiting again uh before uh before he retries that right before he or she retries that. So that is the way we will be packaging the policies, guardrails, eval access into something called as flow profile and that will be assigned to the uh API keys or uh again to simplify I'm saying it as an API key. So to make a call via the MCP gateway so each call will have an unique identity and that identity will be mapped to this flow profile. Okay. And based on this profile every call uh every tool call via the MCP gateway will be inspected and it will ensure that all these policies are met. If not it'll break the tool call. So this is the critical uh component of any AI or MCP gateway. So the fallback right so it doesn't directly uh deal with MCP gateway but uh as a AI gateway the fallback is a critical part right as I said uh if you're scaling your agentic platform for your consumers uh and you're charging them for using it and you need to ensure the SLAs right so what happens uh your token budget is exhausted or if there is a downtime at any of the model provider you can't uh hamper the uh consumer or you can't break the uh consumer's workflow so you need to guarantee that SLA. So that's where the fallback comes in. So uh if a primary model is not available which is like your GPT then you can switch to cloud or you can switch to Germany. So that fallback configuration will be uh configured uh within the gateways. And next one is about evals. Uh so eval is scoring right. So based on the input given and based on the output that it is received whether using an uh LLM model or without using an LLM model uh we will be determining a score for it and we would also configure that any score less than the threshold value will be considered as failed. So which makes the entire tool call to fail. Okay. So that is where uh uh eval comes in. There are any number of evals within each of the agentic framework and there are separate framework to do the uh eval measurements like deep uh evals. Um and last one is uh discovery. So uh there should be two kinds of discovery. So one is like the tools in the portal you can visit and you can see what are the tools that are available for your agentic platform. The second one is the specification for the tool. So when an AI agent should make a particular tool call. So now this is going to become more complex because the agent is not directly making the tool call. It is going to make a single tool call which is going to federate the tool call. So if your tool specification is not good or if you are not maintaining a proper tool specification then your AI agent have a high chance to miss that tool call and which would result in the loss of proper context feeding into the AI agent. Okay. So let's let's go uh a bit deep into each of this. Right. So the connection management so uh there's going to be one control point for every connection. So meaning uh you can have any number of agent but all these agents will be configured with only one tool call. Okay. And then uh yeah uh only one tool call and that will be federating uh the request to the respective MCP servers and its tool call functions. So you don't or you shouldn't directly directly call uh the MCP servers directly from your agent. So you need to have that uh [clears throat] gateway layer in between. So that's about this and as I said the uh federation can happen via any of these supported uh protocols and access control. So who's allowed to let an A agent act for? And this is a critical part, right? So every agent that you build, right? So be it a businessf facing agent or be it your enterprise agent, you need to have an agent identity associated. Okay? And based on the agent identity, uh the MCP gateway policies would be determined and it'll be applied. Uh so it is it is not straightforward and simple but then you need to figure out a way how you can assign the agent identity and this agent identity is not going to change. So if you have 20 agents all the 20 agents would have a same agent identity and this will be mapped to the users's role. Okay. Okay. So when you build a platform, so if uh as a user may happen and I log into the system based on my role, I can be an admin or I can be a viewer and that role has to be combined with a agent identity and then dynamically uh unique key has to be generated to which a flow profile would have been associated and accordingly all the tool calls will carry forward this unique identity for mayapen uh for this agent and this agent identity uh make a tool call and then The gateway would validate all this and then it will make a decision whether to execute the tool or to block the tool. Okay. Okay. So this is about the guardrails. How do you stop agent from uh leaking sensitive data? Uh so this is going to be uh uh most of the time right this is going to be straightforward. So so in your prompt if you're giving any sensitive secrets right. So sometimes I have seen uh um my team members right? So they would uh uh give in the prompt this is the client ID and secret for this API configure this the in the NodeJS application and uh write a API client. So they will do that but then what will happen if I don't have a proper uh governance layer or a security guardrail the client ID and secret will be exposed. What if the agentic platform uh which I'm building logs all the prompt and the client ID and secret can also be get logged in and then anyone can get access to it from the uh uh incident monitoring uh uh space right so uh you need to have the rules and guidelines to block these PII information the secrets uh and then uh banned keywords so sometimes there there are users who would use some notorious keywords in the prompt right so we have seen prompt injections so this is a key part the guardrails is a key part which would ensure that there is no prompt injection or there is no data leakage of PII and sensitive information uh into your uh ecosystem. Okay. And this is about the flow profiles. So as I said so for each agent and for a user there'll be a pro flow profile that will be uh associated. I don't know whether you can see this image. Can you see? Okay. So this is a uh this is a simple uh gateway for the demo purpose. Right? I'll show you a demo as well. So we create a flow profile and this is a JSON configuration. So which will have what is the model it should use. Okay. And then what are the safety rules, what are the guardrails and what are the uh uh quality uh bars which is like the eval score. Whether it could be in the form of completeness, it could be in the form of bias, the hallucination score. You can determine all these package it and create a profile and then associate with your agent identity. So now what happens is for every agent call to an MCP the flow profile will be identified and it will execute all these uh policies ensure things are uh fine and then it'll allow that tool call. If not it'll be blocked here. Okay. And you should have different flow profiles for each of uh the groups within the team. You cannot have one generic profile created and associate that with every agent and every team. Then yeah uh whatever MCP gateways is uh you are on your own. So things would break. Okay. Uh okay. This is the resilience right. So what happens when your AI model goes down? So uh again as I said right. So when agent makes a call it prefers the primary model and then it needs a fallback right. So uh it is about that and these are the evaluations. So whether the answer is correct. So within the gateway so there will be a a simple LLM model that could reside and uh based on the input that is given to the tool and based on the output that it receives. So you can configure uh what are the eval scores that it needs and you can define a threshold and only if the threshold is met you can allow the tool call to pass or you can fail that tool call. So you you can determine any number of such uh evaluation criterias. Okay. and that will go into the evaluations uh section. Okay. So this is about uh tool discovery. So uh the agents need to understand what are the tools it can use. Uh so as I said there'll be 20 agents but all the 20 agents would have only one uh uh MCP that will be configured right but the coding agent doesn't need a call to a jura server okay it needs only a call to the GitHub server. Okay. So then how we would uh ensure that it is calling only the uh GitHub MCP servers or any MCP server that are related to coding. Right. So it depends on the tool uh uh descriptions and the schemas and the way how we configure the flow profile. So that's where the tool discovery comes in. So the normal open API spec as I said wouldn't help. And this is one of the tool browser right. So how many ever MCP servers you configured for each of the tool it gives you the description and the method name. So with which as a human we can also understand whether it is needed for our AI agents or not. Okay. So this is a technical architecture how it is built. Okay. U so there [snorts] okay so the primary consumers would be AI agent and all these gateways would have a dashboard uh and there could be some API and peer agent clients. So since I said replace it with an AI gateway. So the primary consumers for this would be AI agents and the MCP clients. Okay. And there'll be a control plane. So whatever you see in the previous slides, right? Between the agent and between the MCP, there is a layer which does all the policy creation, the flow profile creation and applying that for every request, right? And that is the control plane. Okay. So every tool call will be inspected in the control plane and then it'll make a routing to the respective uh uh A2A agents or to the respective tools or to the models. Okay. And then for every call there is going to be a uh governance uh layer as I said it will be inspected and then we will be applying the policy governance. Okay. U so this is the highle architecture and uh whatever you configure right. So the MCP gateways is going to have a data plane as well and be it the flow profile or be it the uh guardrails or the rules that you create everything will be persisted in the data store and even if the MCP gateways restart right it should uh start from the point where it left okay so uh that's where you will have the data store so most of the modern tools like Kong or uh APG gateways they also have an AI gateway and MCP gateway which has all these out of the Okay. So, I'll quickly show you a demo and then I'll come back to the slides. Okay. Hope it is visible. Right. So uh this is a simple API gateway or sorry simple uh MCP gateway or an AI gateway. So just prepared for this demo. So this is not an enterprise scale uh MCP gateway. So it is the dashboard. It says like there are totally three MCP servers that have been configured and it it shows me what are those MCP servers. Uh so this is for the demo purpose. It is called as eco MCP. There is a playright MCP and a Google search MCP. And it also lists me how many tools each of these MCP servers are uh uh MCP servers have and what is the pro uh protocol type it is stdiodio or uh HTTP. So it lists that and it also gives me the le uh recent events. Okay. And all the three servers are connected. So if I go here this is the list of MCP servers right. So which is already configured and uh it is connected. if I have to create any new um or add any new MCP server to the gateway. So I could just uh do it here. So right now I don't have anything handy but normally you could fill the name uh the transport type whether it is HDIO or HTTP or serverside streaming uh and then you can uh give the command uh and the environment variables. If any authentication is needed, it'll go via the environment variable and all this is within the gateway. It will not be exposed to the agent. Okay. Uh and this is the tool browser. So all the tools across all the three MCPS are listed here. Um so it's there is a Google search uh uh MCP which has analytics, compare, the exact tool name, the description. So everything is available. So this is the key part with which the AI agent will decide which tool should I use. Okay. Um so this is the guardrail part. I can create any custom guardrail. So uh let me so this is a a a guardrail where I I have to block if the request contains any PII data. So I can uh just give the name uh and I can select it is a PI data both in the request and in the response if there is a PI data block that request. Okay. So similarly you can configure all the rules here and then comes the flow profile. So I'm just giving it okay hope it is bit visible right so this is where you package all the policies and governance together that has to be applied uh for every request that the user makes or the agent makes right it starts with guardrail so there are three guardrails that have been configured there is a model cascade the primary is 5.4 for secondary is 5.1 mini and tertiary is clot sonet and eval thresholds I just configured to eval threshold if the faithfulness score is less than 7 and if the hallucination is less than 0.2 to it is considered as good. It'll allow the tool call. If not, it'll fail the tool call. Okay. So, now the flow profile is created. Now, I have to associate this with the uh agent identity and the uh request. Correct? So, so that's where it comes in, right? Um so, let's say like your agent is running somewhere and then it makes a a federated tool call via your MCP gateway. So, you can create an access key like this. So when you do that you can select the profile. So the these are some three different profiles that has been created for the demo right testing API days designed to issue generated and PR review pipeline. So I'll use this testing API days and I'm creating this. So it gives me an uh MCP key or an uh access key. So every tool call from the AI agent to the MCP gateway should contain this unique key along with the agent identity. So based on that it will choose the profile which is the testing API days and then in that profile whatever rules are configured it will execute that in sequence and only if everything passes it will allow the tool call and provide the response. Okay. And the model fallback is configured here and this is the eval quality right. So what are the scores? So for now uh we have only confidence, completeness, faithfulness and hallucination. So we can add a number of tools as we need. So now this is configured. So let me show a quick demo right uh okay before that I'll go to guardrails and if you see right uh there is something called as block words. Whenever there is a hi hello the agent will block that request. This is again for the demo purpose right. So this is a policy that have been created. So now I go here I'm choosing the testing API days v1. And the important part is when you create a flow profile every time when you make a change to the flow profile you need to have a proper audit tracing and you need to have versioning. Again this is for audit and traceability purpose. Okay. So now let me uh okay and then there is something called as eoserver right. So this is an MCP call. So whatever I give as an input it returns back with me an output. Okay. So now let let me try this and I'm going to give some um blacklisted words. So eco high if you see right blocked by gartles blocked words high. Okay. I just want to show. Okay. So, I just want to show you this, right? Okay. So when I hit eco API, it's an MCP call. But if you see right uh it would not directly make an uh MCP call to eco. First it will list all the servers. So if you see it is listing listing everything demo, playright, MCP, Google search and then from these servers it will list all the tools and then the agent will decide which tool it has to make a call. Since in the prompt I have eco and some random text it chooses the eco mcp and it is fulfilling my request and after that it gives me an eval score right so completeness is 100% faithfulness is 100% and confidence is 70%. And don't ask me why because LLMs are nondeterministic, right? So, but this is the eval score. Okay. Uh and there is an event log for traceability, right? So, at every time stamp, what is the tool call that has been executed? Uh and what are the events like the server connect and disconnect? So, all those would uh happen here. I think pretty much uh okay, so this is an access approval part. I just uh quickly show this and I'll go back to the slide. U so every time uh at the runtime if we need an approval right so we can configure it here so uh okay so the echo tool is approved here let me reject it and if I go to okay if you see right I trying to I try to run echo But then uh it has been uh blocked. The access has been rejected because I have removed the approval uh for this particular MCP call. Okay. So this is a quick demo and then I will okay. So these are some screenshots of the demo. Okay. So uh this is some value proposition that MCP gateways uh delivers to the enterprise right. So be it consumerf facing agent or be it an internal set of uh agents. Uh so it shrinks the attack surface or it reduces the blast radius. So when you federate all the MCP tool calls via an MCP gateway the blast radius if at all if something notorious happens it is going to be within that small ecosystem and it will not impact across multiple agents and you will have a full stack of audit right for every tool call for every request at every time stamp you will have a complete uh audit trial with which you can at any given point in time look back and understand what happened whether the quality is deteriorated whether the goal has been completed properly. So you can use it for your uh debugging purpose and at the end you get a complete confidence right without this think of a situation right you will not have anything to monitor you you cannot see what the uh guardrails and policies is the agent at any point in time can call any tools and it gives some random output we can't even identify why it behaved it in a way we know LLMs are nondeterministic but at the same time at least with what is deterministic we should have a mechanism to understand what is happening So with this with AI AI gateways and MCP gateways so you will be able to ship your platform the agentic platform with complete confidence right and it also helps uh misbehavior structurally impossible. So this is just the starting point right the more and more new stuffs come uh comes up within the AI ecosystem things will mature more and uh things would change uh to to bring up more uh level of securities right so with this uh you would be able to make your system structurally impossible to have any misbehaviors uh and that's it I'll stop here and I'll take up some questions [applause] yep I think it's a very elaborated presentation and also implementation. Thanks for that. Uh the see we also doing the same thing at our institution. The the biggest challenge what we have is that see if the agent and user are within our infra we are good to identify them because it has a relationship but if you expose our MCP endpoint to the external agent or external vendors how we can make sure that how we'll identify the agent and how we'll make sure that what user of that agent is using it so that we can identify them that's a challenge what we have it so you have anything you tried on those things >> okay so we haven't tried but to your question right uh So whatever your question is it is the inverse of what we were discussing till now. So uh what we were discussing till now is an internal agents internal gateway. Now if you want to expose your MCP to the external world still the same thing applies instead of directly you expose your MCP uh it could be an npm package or whatever it is expose it via a gateway. Uh we'll take a step back right with APIs few years back how did we expose our APIs uh to the external customers? We did not directly expose it. We had a uh API gateway in front at least what we were doing is we had an uh APG or Azure API manager we will make it a public API and we will have a stringent uh policies to the incoming request for this API. So similarly when you want to expose your MCP so have a gateway in front of it and expose it via the gateway and whatever policies and governance you want to enforce it enforce it in that layer. So tomorrow if you want to make a change or create a new version of MCP it will still happen behind the scenes to the external customers it is always that MCP we have API gateway so mean MCP gateway we have it the thing is the uh we actually have challenges in that how to identify the agent the external agent how we can quantify okay is the right agent and the right user using the right agent so that I can allow my request the next layer we have a gateway but internal in infra we have a user identity we have agent identity we and cross check okay the right agent right user >> okay >> so each of the uh AI and MCP gateway provider does this in a different way I don't know which uh gateway you are using but ours is a homegrown solution and every agent that we create right with the framework support we assign a agent identity and that will be a dynamic agent identity and we will validate that since in your case right so you have an MCP gateway but if the incoming requests are not within an agent identity you need to enforce the external customer to create an identity for the agent. >> Okay. But in your infraos if you want to use your agent or MCP gateway I'm a different allgether platform agent agent can be anything it's not be the same platform same in so then what is the center point where actually we can inversely identify okay this is a agent identity is unique >> okay so it should be whoever is building that agent and we need to create a dynamic unique identity >> but how to share with us okay >> pure is is external. No, it's a vendor. >> Yeah, >> you can actually block them. >> No, but we want to provide a service to that agent. We can't block. >> But without an identity, you should not >> identity then it doesn't make >> how we can make that handshake. No, because I want to provide a service but I want to make I want to identify my customer. >> I'll go with this answer because when you expose an API, you do give them an identity, right? that is we do with what two that would o is not identity it is for your authentication purpose agent identity is completely different >> we do with the client ID that's what your identification there >> uh you need to think through that oath would not satisfy an agent identity requirement >> no that's what I'm saying in API case we go with client client ID secret we do that one >> now but what is in the agent world how identify them >> so what you are telling is you want to expand the agent usage inside >> outside the organization >> yeah outside the organization but still you would need an identity right otherwise >> that's what how that handshake happens with identity that's what I'm saying >> okay so I I I don't know about your use case but what we do here right so whenever an agent makes a call to the uh MCP server that single MCP server in the request uh header params we would get the agent identity and then we would also get the flow profile okay the agent identity then will be validated within our platform is this the identity that has already been registered in our gateway Okay, whether it is a true agent or it is a notorious agent and then we will decide whether to even further proceed or not. So [snorts] the agent identity can be two things. One they can have a identity created registered with your gateway or you can generate and provide an identity which they will map it with their agent. It can work both ways. >> That registration how we'll do it that's what I'm to say. >> So the registration can happen again uh through the portals right? So when you create a profile so I I showed you a profile right? So when you create a profile, you could also assign the agent identity only for this agent uh this flow profile has to be uh associated. So you could do that. It is the way of customization. It is not completely mature. It is still uh budding. Actually the space is still budding. But >> because you also within agent we using the keys to identifies right the keys is >> associated with agent ID that is just API key itself. >> Correct. >> But is it secure enough? >> Correct. >> Is it secure enough? That's what I'm saying because still key can be misused. It's just it's a guy key is just a UID right? There's nothing associate more than that. So suppose if I expose API, I don't prefer to expose API key because it's not sufficient. That's the reason we go forward to one more layer because API key is not good enough. So this is a simple example as I said right I'm just generating a key and then associating that for a tool call but when you do it at an enterprise scale so whatever your enterprise standard is so along with the agent identity as you said right if you want to have a client ID and secretbased token and associate that with the profile you could still do that it depends on how your enterprise cyber security standards are this is just a demo it's a very simple demo so don't go by this >> no for thank you >> okay >> yeah one quick question >> yeah sure >> so what controls become mandatory when agents consume APIs at scale. >> Okay. So you're asking like what are the controls that are needed when agents access the APIs >> at at a large scale. >> Okay. Okay. So the primary control right uh when an agent want to directly access the API it has to be again wrapped with an MCP uh tool call. So what will happen is your AI agent will make an MCP tool call via and gateway and the MCP will make an API call via an API gateway. [snorts] So whatever principles that you follow for the APIs in the API gateway layer would still be sufficient. So here what we are discussing is on the MCP side. The MCP can make an API call, it can do a file operation, it could do an GitHub call, whatever it is. So the MCP gateway plays in between agent and the MCP. But in your case, if you want to have that API, it is between the MCP and the API, we have the API gateway and whatever uh the rules that you have today to make it uh scaled for any number of consumers. So you could still apply that. >> Okay. Uh and the next one is uh what governance failures are appearing in early MCP deployments. >> Sorry, come come. >> So what governance failures that you have observed in early MCP deployments? >> [snorts] >> Okay. So, one of the major flaw that I see or when I talk to people is they directly uh integrate or they directly configure the MCPS with the agent without using an MCP gateway. And I have also seen people who have uh uh who are using the MCP uh gateway portal and they configure any number of agents where people can download and associate with the agent. Right? there is no governance or even there is no validation whether it is from the right right uh uh source or uh >> MCP gateway is the solution is what you >> MCP gateway is the solution >> okay how do >> sorry uh we can take the questions offline the time is up I've been getting questions thank you we can take it offline okay thank you thank you thanks everyone Okay.