Zero Trust Security For AlwaysOn APIs | apidays India 2026

Hello everyone. Uh thank you for joining this session. This is a demonstration on zero trust security for always on APIs. U my name is Kamalika and I'm the founder of stack llp uh 10 factorinfra.com. Um I've been a uh infrastructure uh builder, designer, developer uh more for more than a decade now and I primarily uh build infrastructure that is compliance and audit ready for uh regulated industries such as banking, financial services, um insurance etc. Um this particular uh session will be a real life experience of implementing zero trust security and building the infrastructure um for from from one of my client projects. So this is a this is from a real life implementations although I'll be um not sharing the exact uh you know details. I'll be uh keeping uh things as uh standardized as possible because I am uh under NDA and all those things. But this is a real life implementation uh from a bank. uh some of the key features of this implementation uh and why zero trust security was uh such an uh priority or like uh requirement for this implementation was that this was a 100% cloudnative digital banking application. Um what does that mean is um this bank did not have any branch. So they were all operating from a mobile application from customer registration to transaction. They literally did not have any branch or any office. They had uh one satellite office but it was not used for any banking operation. So it was a it is a it was truly a 100% digital banking application. Um this was from a this was an Indonesian bank in Southeast Asia and um they for that reason they had a SLA for uh of 99.95% and that is why always on um infrastructure or APIs that uh my demonstration is about and we'll see why it is called always on. So 99.95% uptime was their um written SLA to their customers right and partners. Um being in Indonesia they had to adhere to compliance and regulations and being a bank uh that meant additional compliance and regulation. Um they had OJK which is uh you can say similar to RBI but it is much more stricter than RBI. They are the financial services auditor of Indonesia. So they had to comply to get the banking license. They also needed ISO 27,01 for information security. Uh in some of the places they needed SOC2 and BI stands for bank of Indonesia approval. Um and these included some of the key things that included was PII data localization for uh this bank. So because Indonesia has a policy that any kind of payment data or or the PII data for any payment or any other services had to be within uh their country or if I may rephrase that data must not leave the borders of the country. So that is a very strict regulation of Indones in Indonesia for PII data. Plus uh Indonesia being uh uh you may know that it is one of those countries in the ring of fires right it sees very frequent uh earthquake and you know tsunamis and all that. So it's a very disasterprone region. So that's why disaster recovery compliance is very important for Indonesia. So whatever SLA they're defining they have to comply to that. So they have to prove they have to literally do a dr and prove with evidence that they can comply to it because uh in in their region uh anything can happen due to the uh you know kind of geographical uh disasters that uh usually they face. Um plus uh this bank had a uh you know kind of self uh identified uh kind of feature which say quality and performance. So they wanted to be very very uh upto-date in line with the modern day technologies not like a traditional old age bank where you know you just give transaction facilities and all they wanted to have super quality and performance so very snappy app that's what they they liked to call plus scale on demand so they were hoping uh to onboard uh customers to up to around 1 million customers 1 billion customers uh and these will all be using their banking application Right? They will be setting up kiosk in malls and you know different locations and then they would just uh their way of marketing was just come and get registered and you'll get benefits and like so they were expecting huge amount of customer onboarding because that was how uh they were targeting market. So scaling on demand was also a requirement. Now it is important to remember these things because this will be part of some of the key architectural things that we had implemented uh in the bank and these will be the background behind those architectural decisions. So always on APIs why uh I call them always on APIs because uh 99.95% SLA 100% digital. Now if we look at 99.95% SLA some might say okay it's not 100%. But mind you for a bank that that what that means is uh like this particular bank as I said in a day they were expecting around million users across Jakarta and ac outside of Jakarta and you know they were they had done a kind of marketing in that sense and that was their success criteria uh to make this banking uh application successful. uh they had to deploy an audit ready production environment uh in minutes they have to be I mean they have to be ready with that you know at any time uh some auditor can come in and say that show me that you are this so they have to be always audit ready it's not like once a year they will do an exercise and then uh you know it it was not like an audit ritual it was literally they had to be prepared for it u now why always 99.95% actually is an always on because That translates to disaster recovery with 4 hours RTO and 5 minutes RPO. So I don't know how many of you know the terms but 4 hours RPO means let's say disaster happens your system goes down you have only 4 hours to get your user endpoint up and running. Let's take say SBI or SDFC or any bank right it is running and suddenly something some an earthquake happen the user loss connection it you are uh given only 4 hours to get it up and running and this 4 hours is for the entire year not that day. So if you exhaust that 4 hour in that disaster so for the rest of the year you are not going to get any uh you know hours that's the knob that's that's what means like uh in a year yearly annual cycle plus 5 minutes RPO. RPO means um let's say uh your uh users are constantly transacting creating uh registering accounts etc. Now there's a lot of data reads and writes happening right? So there's a chance of data corruption um recovery failure and things like that. So let's say disaster happens your data center goes down and things like that. So you can the business will only accept up to five like till the from the time the disaster happen uh till the recovery happen only 5 minutes uh you know you can you can expect data corruption for that 5 minutes the delay you know uh but you will have to recover it from the point when you had uh when it went down but you can adjust let's say okay 5 minutes of data we lost but we like say 11:00 a disaster happened so you can count from 11:05. So something like that. So you'll have to be extremely prepared for it so that your data is constantly replicating. It's live and active active environment and it is recoverable. So that's what makes it truly always on APIs and um because this is also an incident where uh this was not a disaster but it was like a migration. We wanted to migrate from one cloud to the other. So it was not 100% live at that point of time. it was only live to a minimal set of like friends and family customers but in within the just 4 months uh the data size grew to like 3TBs and we had to actually migrate all of that data within four to five hours so like even when we as a bank are doing the data migration we had to follow the norm that we don't cross that SLA so because you know our management would be there when we are doing the DR uh or the translation and the moment it it crosses that they'll be like stop it we are not going. So it it was that strict because that's how that country operates. They're very strict about these kind of regulations which are set there. Uh and yeah when all these things are happening even during maintenance window you want to upgrade a Kubernetes cluster you want to do some kind of certificate installation you will always have to keep in mind that the endpoint should not go down. There should be zero uh downtime deployment at all the time. Now why zero trust here? Now the the particular use case which actually drove us towards zero trust security or architecture or the requirement which later on extended uh to a end to end zero trust uh security uh kind of beyond the policy was this particular requirement. So we all know visa right we have used visa card but visa is also a payment processor. So Visa was their payment uh gateway selected one. So it was a requirement actually from Visa to the bank that if you want to integrate with our APIs, if your services are going to call our APIs, you will have to ensure you have mutual TLS for all your endpoints internal and the ones which are connecting. So what does mutual TLS means? It means two-way SSL. So we know this standard TLS or client connects to server TLS exchange happens. But in this case even on the server side certificate have to be validated on the client side. So it's like two-way verification has to happen and uh the reason what they cited for us was that we want to prevent any kind of man-in-the-middle attack. Now I'll come to that what is man in the middle. So it was typically a third party integration for them and also for us right we we were connecting something someone else's API uh and we were we we did not have a direct uh like a link like they were not on the same uh the traffic could go over the internet basically. So it was HTTP endpoints, HTTPS endpoints on the both sides and they will have to exchange security certificates on both sides not just the client um you know validating the client. So the even their request should uh when they land on the bank it should be validated and they wanted it even for our internal services like say we have a um transaction history micros service and then uh we have like um uh what could be like another payment service a microser now these two services are talking to each other even then you have to use a mutual TLS mechanism that was the requirement so they wanted us that you show us that within internally Finally also you had zero trust policy then only we will approve access to your APIs. So that kind of drove us towards zero trust security because before that we were doing HTTPS on all the external facing endpoint. So we were always thinking about who is going to access the bank. We were not we were all okay everything is private. So within our services it's okay because that's a standard usually people miss that right like uh one microser sitting between two name spaces fine at8 is fine but this kind this requirement actually forced us to think that even uh within our services there can be some kind of breach unfortunate uh situation where you know some microser can send some wrong uh kind of access request outside and it can eventually ham ampera third party and vice versa and and that's why we uh you know u decided to follow zero trust across every uh communication endpoints. So this is particularly what man in the middle attack looks like. Uh this is a standard client server communication. You have https usually on the server side and then [snorts] it the client uh accepts it validates valid certificate and it's good. Now an attacker can actually snoop in uh in that because everything is happening on the internet right it's free uh nobody is restricting or you can't encrypt the internet in internet the communication is actually the channel is actually uh uh [snorts] open no matter what you do. So they can uh sniff in between and then um you know they can send malicious requests to the server uh which can in turn hamper the server. Now another reason beyond this man-in-the-middle attack uh is another uh attack vector was also DOS attack. Now this being a mobile banking application the API was open like we could not restrict the API saying that it is only available for these IPs or this building or that building because users can users are basically using their mobile data right mobile data Wi-Fi they will be anywhere they will like I said they will be standing in a shopping mall where the kiosk is there they'll be opening their bank account so uh attack was very likely because the endpoint was open to public now and because uh DOS attack they may not be able to break our firewall but if they launch enough attacks to the endpoint the endpoint would eventually give up and go down. It's a standard I think most of us know by now what a DOS attack is. So but this is also a reason why we wanted a solution wherein even though our uh endpoint is exposed to the internet we would know what kind of traffic we are sending from our endpoint to the API gateways right we didn't want like a blanket thing to go like everybody come in and access and that is why a lot of uh infrastructure decisions was taken um so basically we wanted no implicit trust nobody should be uh gaining gaining trust without a reason. Basically, even if he's an admin, he's a lead or whatever from user uh from internal user to external user, there should be no implicit trust. Everything should be verified, validated. Continuous verification should be there. Continuous uh you know verification does not always mean approval and uh you know thing but at least it it should be identifiable who is accessing what and why. Least privilege access across all infrastructure except for the services and applications talking to each other. Human to services, human to application that should be as much least as possible. That means no root access, no you know emergency level SSH access to a box and fix this thing. No, we kind of uh set that discipline within ourselves that we ourselves will not do that. We anything that has to go in has to be trapped. anybody that that can mean anybody from a uh who who is at a higher position to an engineer or developer who may not be responsible to you know answer any uh you know investor or anybody. So, so that was a practice that was a policy that was set from day one securing third party integration from man in the middle attack as I said. So, visa was not the only third party that we were accessing since it was a bank. It also it was also storing identities. So, like in India we have Aadhaar Indonesia have something called as dupin. So there that was kind of their uh you know country identities that were getting. So we were also connecting to that which was a government uh site. Now there were a couple of other government plus private side that we were connecting plus there were SAS applications uh you know like uh we were uh using a multiloud SAS for our uh logging and monitoring uh but it was not in our cloud or our VPC or nothing. So there were a lot of third party integration happening. All those have to be pro protected because any one of them including us getting u you know exposed or getting breached would hamper all of us and that's what we are seeing today right so many of the cases or the breaches over the last two years that we saw uh much less of them were for the actual provider. It was somebody who was integrated with them and they had a breach which caused the primary source like even the cloud flare or recently there was a breach with hugging face and open AI which was like somebody else but they are connected with so it always uh and that is why even with the compliance and regulatory standards that was mentioned in the previous talk like 42,0001 or 27,0001 [snorts] they have in in included third party uh supply chain security as one of their controls in the latest update post 2022. So if you see the previous versions they did not uh kind of had a control for that but in the recent version each and every one of them including the AI one plus the information security they have very specifically said you have to make sure that your contractual so they they tell it from a contractual angle but from a security part when they come in they would ask question like how are you protecting like contract does not always mean a paper uh contract it also means the application level contract Now zero trust architecture, zerorust policy these all includes uh you know the whole thing uh together says about zero trust security. So let's look at some of the actual implementations uh I'll be showing more of architecture and some code snippets uh example ones. So first uh was the always on infrastructure. Now it's a derivative of the architecture. I show the actual diagram. uh the actual diagram is quite messy because that was built by a human. So uh this as you see uh what did we do for always on infrastructure to achieve that 99.95% availability. So by now I think most of all of us are familiar with these terms right availability zones and cloud and VPCs. So it doesn't go beyond saying that you need to have multi-AZ setup right for high availability. Now this one since it needed uh two uh uh you know 99.95% availability it also needed a multi-reion setup. So what is multi-reion? Multi-reion does not always mean multi- country right within country also you can have region like say in India AWS has one uh uh region as Mumbai one is Hyderabad. Now say something Mumbai gets hit with tsunami god forbid uh if let's say it happens. So then you if you have things live replication happening in Hyderabad you can switch your traffic there. Now in this case it was interesting because when we started this development of this mobile banking application there was zero cloud in Indonesia. Um now we had to think about uh what to do if that data center goes down. uh what we chose uh was that at least uh in Indonesia's um you know policies and regulations what they say is the DR site let's say your primary site and the secondary site they should at least have a 60 kilometer um you know kind of minimum starting 60 maximum can anything why 60 because let's say they are so prone to earthquake is that uh within city itself like within Jakarta itself uh if you keep things like uh just everything within a same technology park or a data center it will fall. So what they were trying their best is that let's say one thing is in Jakarta, one thing maybe 60 km away from this from where that data center is, right? That's a minimal because it's also an island country. It's not like India. It's not it's not a big landscape like India. So in this island country to have another city mean meant another island maybe Bali or Java. So that was uh like even today they don't have that setup like it's been I think 5 years or 6 years since then. The only data center you find is is in Jakarta and this is the case with many of the Southeast Asian countries like Singapore, Jakarta. Now some clients did consider uh a data uh recovery zone in Singapore, Indonesia to Singapore but we could not do that because Indonesia remember had a very strict uh OJA policy that data must not leave their land. So they have a very strict one. Now if maybe like I don't know about Malaysia and others maybe Singapore can uh host something outside some of these uh things but yeah multi-reion was very important so that one failure uh one system fails uh it can uh always go to the other side. Um the other thing was the it says global load balancer but what is actually is the like load balanced entry point you can say you know it's not typically a load balancer there are load balancers but load balancer was not the only thing. So basically [snorts] load balanced endpoint across uh these regions. So in this case um we had a combination of cloudflare API gateway and um the usual cloud load balancer. Now when cloud load balancer we had different types of load balancer. We had load balancer which are external that is exposed to internet for the ingresses and the data data database was not exposed the ingresses which will be accessed but we also had internal load balances. So the practice that we followed was that even if you are exposing two machines don't take it lightly saying oh it is just a machine we'll just expose the IP. No, what we did was we always exposed uh an endpoint with a load balancer. Now that incurs some cost but it was a bank so they had money but uh uh uh I would recommend for an enterprise who is aiming to reach that kind of scale and performance uh always expose things with an intern with a load balancer even if it is an internal traffic. So for for for some some people might say like things like VPN right VPN client servers some say VPN is not uh important right but you will not realize VPN is not you will realize when VPN is important when there is a production issue occurred and your engineers are actually stuck at home and they are trying to login and VPN server suddenly went down right and you don't have a kind of a traffic translated to the other VPN server so is this is an example like sometimes some of the things like uh maybe consider trivia trivial it's not super emergency but we should always think about uh when crisis hits you need to make sure the things are accessible the other thing was uh yeah SD buckets so they had a lot of assets like banks have a lot of identity you know images and proofs and things like that so those are also load balanced um and and we also set up everything with a heavy monitoring system monitoring was set up at every level right from application system infrastructure in In fact um we had something called uh we use something called dinat trace I don't know if you guys have used dinat trace so dinat trace does a very uh uh like system till application level instrumentation like to the level of which java uh exception happened due to which user request to that level of so we needed that kind of thing because that kind of uh debug capability was necessary plus we also needed security monitoring to collect all the audit data For that we had another system which we built ourselves uh which is the science system the security identity and event management system. So they call it Siam uh we did not use any third party application we set it up ourselves where we collected all the data to a cluster to any cluster and that data was uh you know constantly used by our uh security teams uh for preparing. Um so yeah this is pretty much what the overall but one thing which is missing there. Um so there's the load balancer endpoint was uh the all the endpoints are load balanced the multi-AZ multi- region networking. Uh there was HA clusters for databases as well as applications because of the high availability criteria. So they there had to be replication across databases. Uh live replication not a backup. Backup recovery was also taken by the way because that is also a regulatory requirement that you have to have like some can argue what is the need of backup since we're doing replication but no backup was also needed which had to be tested. Um horizontal autoscaling as I said there will be a like you know wave of customers who will hitting hit that. So we we had no scope of waiting uh to you know till we till the requirement comes in and then adding servers to it. So it had to be autoscaling and we always did horizontal autoscaling because you also had to you know conserve the resources so that if there is zero customers let let the no the pods come down so that it does not over consume stuff. uh as I mentioned this and yeah this one is missing in the previous diagram dedicated private link between these sites. Now why was this needed? This is the actual diagram. This is not very clear because it was uh it is actually created uh in from multiple sources. So they had a combination of cloud and onremise. Now why onremise? Like I said uh Indonesia uh finding a cloud which which had data center across you know 60 km away was very difficult at that time. So an interesting incident which happened was um we were evaluating GCP uh GCP had launched their first uh the actually the first clouds were Alibaba cloud. So it first went on Alibaba cloud. Alibaba cloud had a data center which was um kind of 40 50 kilometers away two data center like as their AS were literally uh like proper data center. Now for uh performance and other feature reason we wanted to move to GCP. Now in GCP they did come to us and say that we are OJA compliant. Um if there are any GCP folks please don't mind because I'm just uh saying what I faced. um uh this uh they they were the first one to launch uh like the global public cloud. So they did say they were OJA compliant and they had three availability zones one region right now uh we thought uh and this was also a mistake from our we thought GCP is same to Alibaba cloud or AWS which is not all clouds networking was uh are different. So when they say GCP says availability zone, it does not necessarily mean separate data center. G in GCP if you have to truly get uh high availability you will have to go for regions which is they say it is as zonal right for them it is a well zone. Now in this case when we actually did the audit of the data center it turned out that uh they though they had three availability zone all those three were actually across two buildings in the same tech park right so uh had we not gone there to physically review um we could would not have caught it because yeah like no cloud will come and tell what they so we they did not tell us we actually went to like we had to audit it because we had to set up our uh private links between GCP and some of the other uh you know uh between GCP and Ali cloud so things can move faster because and of that uh we had to take a decision that uh we can't this is not an HA for us uh we have to have a secondary site so uh we had a um you know onremise uh building where uh temporarily we landed all our third party connections there and from there it was gone to the cloud and then we had private link between GCP and our that building and uh one link was also set up with Alibaba cloud so that in worst case scenario we have two fallback option now when we were setting up that we could have done it easily with VPN side to side VPNs but that was not enough for both for man-in-the-middle attack VPNs do not also do not protect man-in-the-middle [snorts] attacks like VPN is also through internet so you though you are doing like I said right visa had asked only two-way authentication based zero trust they did not ask us you need to have private they actually did not ask us but When we were doing all these exercises, we came to know that even on the internet VPN toVPN, there can be man-in-the-middle attack because it was still VPN can still be breached. So that's why a private link anything leaving or accessing our site there was a dedicated private link by private mean literally fiber optic cables were laid from our building to their data center and then uh our traffic would go uh between uh Visa and and those and that building. So at least [snorts] we know we were on a trusted network. And in GCP there were one one third party uh service which was also hosted on GCP. So what we did internally was that we made sure that we we were operating on a kind of a private uh service link between GCP and that our third party service because both had same private cloud provider. So it is possible that if you are within a AWS you and one of the APIs that you're connecting you can actually ask for the cloud provider itself to set up a dedicated link so that you have uh and it also gives you better speed. So because all the bandwidth will be used by you and not the uh internet. U now in zerorust policy we use zerorust security as code. Everything was codified when our infrastructure was created. we use terapform and in some places anible and some places python scripts for data migration and things like that. So segregated networking uh these were some of the key things that were used for specifically uh building a zero trust infrastructure. So segregated networks I'll show you what it looks like. IP whitelist and not blacklist. So first we denied everything, blacklist everything and then whitelist and uh only authorized traffic would be even between our applications and databases and systems. I'll show you the architecture how it looks like. uh raise alarms even when internal something fails like even if there is a failed attempt just raise an alarm because too many failed attempts uh this actually happened that there were too many failed uh uh access requests from one particular VPN that was coming up later on it turned out to be some uh you know wrong application trying to hit the other. So it was good. It was not some external attacker but it was good that that alarm was captured and we were alerted that something in configuration was we had not fixed like we had kept it open. Usually sometimes it will happen when you are building something from scratch. Uh role based IM everything was role based which meant no root access. So first thing we did when we created a cloud account was disable the root access. No root access to any system admin any infrared anybody. Then multiffactor authentication was must. Um password policy had to be very strict and to be rotated every 60 days and deletion policy. We also had a very uh clear set guidelines for deleting users who have already given access. So onboarding and offboarding and it was always done using a pipeline and with a script like the Python script I mentioned actually we had written a Python script which will call the VPN API and run it. We were not we did not do it from the UI. Many times VPN uh kind of management is handed over to the IT operations or some people who use the UI to add user because that's faster, right? Add quickly add user. But we made a policy that no even then um that trivial task had to be all traceable in our uh pipelines. Um and uh for that and that also came in handy when we had to clean up a lot of user licenses. So we can run the deletion script and then all the you know disabled users uh we could do enable disable onboard offboard through a pipeline. It was easier and faster. So yeah segregated networking this is what we did like we blocked everything. We had different VPCs for administration and different for the application. Administration VPCs usually will have all your management system like CI/CD, monitoring, auditing system, the centralized system we will actually capture data and monitor it. Uh and then in the application side you had segregated networks separately. The ones which will be exposed to public will sit in the public network the and only that will have access to the net gateway. The second one is uh private uh you can say private or protected like it has access from the web network and towards the data network. So data network was completely private own no outgoing uh only incoming. So the second slot that you see it had whitelisted incoming and outgoing also it can it can go to the internet because it can connect to because these were this is where our um microservices were sitting from where it will communicate and port level also was all uh whitelisted uh like uh no blanket all TCP all UDP and all that. So one thing I wanted to mention here is that how we also did uh block all the uh default traffic. So one of the things that was very important is that in the in in any cloud when you go and launch a there is a default VPC always created. Many [snorts] a times we ignore that default VPC because we don't use it. But that default VPC does actually has allow allowed to everything. Now make sure when you are inside your crowd account you make sure that you delete that VPC before you are creating your VPCs. Do not go with the default VPCs and even after uh sometimes you are sometimes you might not get a fresh cloud account. You might be setting up uh you know and an existing setup which somebody else is go and check in your default security group with your main VPC whether uh you know you have this kind of uh security groups allow you know created or not. create a default security group which had no nothing no rules nothing so that that will by default deny all security group traffic otherwise what happens is we might be denying all security groups on our VPCs but the default VPCs which is created by AWS and GCP they will still be allowing it so that there is a loophole that stays there and uh uh we had different gateways for internet and and the net and as I said separate private private protected and public subnet. So this is teraphform code. Um we use terapform from infrastructure automation. Uh anible was used only places where terapform was not available. So anible was more for uh software or SAS automation. Uh the other so this is uh the post migration to GCP. This is GCP diagram. So at a later stage we actually moved to GCP. Uh as I said a single secure endpoint was created using Cloudflare. So Cloudflare was the Cloudflare IP was the one which was registered in the DNS which the users were using their mobile uh to register to the mobile banking application. So why Cloudflare? It gave us couple of features very specific feature. First and foremost they had a data center within the country. So we we were uh in compliance and they they had a good DOS protection available. Now you might be using Akami or AWSS shield and GCP also has something I forgot that name but it is mostly for DOS protection implementing VAP roles so that that whitelisting happens uh on application URL patterns. So um on the URL that are being exposed some malicious actor can kind of tweak the URL and try to launch a proxy attack and can bring down our server. So that is also one kind of attack for which we had to do URL web filtering and uh web filtering web masking all those things OS has a you know like 10 or 250 rules you can apply that and Cloudflare has a system where you can apply these rules. Now in the recent compliance updates like ISO 27,0001 they have actually added web web masking and web filtering as one of the controls which is very interesting that even the compliance and regulators are slowly uh you know adopting the security features which are needed in the which were always considered in the application side and usually compliance did not pay much attention to that. Um so as you as you see that there are segregation in all level that is also another zero trust uh thing not everything can access everything only uh a certain applications which needed the certain databases they will be able to access one application if it was not meant to access the other database it was not allowed in the database. So databases also had a very clear IM. So everything uh you know across all these uh layers rows that you see uh IM policies are applied and uh no admin access access allow allow things those things were completely banned only selected you know role create specially created roles were allowed to have access. Uh yeah so this is also similar uh this is the deployment diagram across that admin projects and the application pods how they were uh run using terapform as I said we use terapform so terapform uh usually uh was run from our cicd pipelines in gitlab in so gitlab runners were sitting in the admin projects from there we would initiate pipelines and then we will create it and those gitlab runners were also using certificates and authentication mechanism and no uh secrets were all stored in secret management. So they were actually calling them in runtime and nothing was stored on the um at rest. So zero trust uh architecture like I said like never trust always verify everything should be ver every request au every authentication and authorization that happen has to be uh validated before it is allowed plus there has to be continuous monitoring at every level so the both from the in API level application level as well as from the infrastructure level what happens is most often when you talk about zero trust policies and things we always we only think about authentication right only on the application level we don't think about the infrastructure that in infrastructure it's it's just ports and protocols right there it is very important that uh whether you are what you are allowing and whom you are allowing to so there you do not in the packets that you go you cannot really authenticate who is going so you have to be very careful in the infrastructure level that's why you have to make sure that you know you are filtering everything out when you are sending traffic to your application. So that's why DOS protection and the rate limiting is important. Uh so this is what uh man in the middle attack uh does as I showed in the previous picture and what mutual TLS does that like I said in normal TLS client uh request the server and it gets validated at one side but now it is like both side. uh in our case like uh I said we had to do it on within the infra within our systems also. So we used API gateway called tyke right we did not use AWS uh sorry GCP or AWS one I think it was not available there at that point of time in Indonesia. So what we used was something called STO I don't know if you guys have used STO service mesh. So we used HTTO because we were using Kubernetes uh you know GKE and EKS cuz it was easier to set it up. So we used mutual TLS in HTTO our own with our own certificate servers uh to not just authenticate with Visa which was the use case scenario to like Visa had requested us to have mutual TLS. That's where we started but that's where we ended up doing mutual TLS within the Kubernetes uh ecosystem as well. Right. Um so yeah so this was uh right anything else I'm missing yeah some of these also needed up another third party as I said the identity provider which was uh used which was a government identity provider but that for that we had used IP filtering and uh private networking Now another thing in the EKS cluster for zero trust what what we are applied is that we had used uh index filtering also so that uh like allow all traffic could be prevented and those uh pods and services could be pre prevented from any malicious or you know blind traffic that is coming there. Uh last but not the least continuous compliance. Um as I mentioned like they had certain compliances like OJK and ISO 27,0001 that was quite you know before the AI era but now with AI coming up this these things that you see they they are changing at a at a what can I say at an AI speed an exponential speed. So today we don't have enough time to sit and take approval or analyze or run you know run PC's and then come up with a solution. These things are changing so rapidly that we'll have to prepare that we will find much more vulnerabilities much more uh you know changes uh caused due to uh you know trust relationship uh because the AI agents will be trusted more people will give more permissions to the AI agents. Now that those kind of situations we'll have more often and that is why these are the some of the global standards who have also recognized that so like in ISO 42,0001 uh is the new standard and the only standard I can say like a framework which is given which came in 2023 for AI management system. So if you read through the ISO compliance, it actually goes much much deeper into who makes the decisions, who owns the decision and who is governing it, right? Uh they don't tell you about what technology or what protocol or what MCP you need to use. They clearly are heading towards saying who is governing your API decisions or AI decisions. Right? But ISO 42,0001 is not enough for AI. If you think that I will only apply ISO 42,0001 that would not be enough to make sure that you have proper compliance, you need ISO 27,0001 also which is the older one for information security. So you will have to combine both of them and make sure that you apply all these uh controls within your setup. So for instance, what I call this is uh compliance as code, right? For example, uh you you need a continuous monitoring of your audit controls. So you need to continuously monitor what is changing. for example. So this is a this is an AWS config rule where you you check whether your policy has all IM policies are read only right all IM rules or uh you know uh your managed instances are being monitored for vulnerabilities or not. So you need to run this is AWS config which you can run continuously in the background in your setup. This is continuously scan for all the controls in the compliance. So having a like I mentioned earlier having a yearly audit or an audit ritual just because your management said that your certification has to be done is no longer enough. You need to have this running just like you run your logger continuously in the background for your application for your application log. You should also collect everything from for your compliance log because when the auditor now comes he will be asking you for real time uh you know uh evidence. he will not give you 2 days or 1 days to collect the evidence and screenshot and go there. So that is very important uh in terms of continuous compliance. So last decade we learned about continuous delivery. Now we'll have to very fast move to continuous compliance and uh in terms of the regional one these are the some of the written government rules that anybody operating uh in those countries have to follow like EU AI act it has already been formalized from this August anybody any business operating in EU has to follow EU AI act when you're doing EU AI act they actually consider ISO 421 also so if you don't know where to start. You can start at ISO, you know, then go to EU. It will be much more easier and NIST RMF is in US. Uh Korea has an AI act. I think Singapore also has one. So most common thing in these um compliances is one. um having implemented and having audited in ISO, I can say that one thing that they require is continuous monitoring, measurement, analysis, evaluation and improvement. And it is a written rule, written clause in all of these ISO compliances, right? It has to be continuously monitored, measured, analyzed and then it has to be improved. If you're not doing that, if you don't show that you are actually vi you know kind of uh uh generating nonconformity with these compliances and you will fail your certifications. Okay. So it is very important monitoring has become all the more important with AI taking this space uh because we need to know what is happening in our setup. So >> can I ask a question? >> Sure. Yeah. So that's all. Uh this is a closing note. >> 5 minutes. How much time do we have? >> You can take >> Okay. So, yeah. Uh I'm closing here. Uh >> if you could go back to the slide where you're showing the load balance certificate management uh procedures. The slide which had >> this one >> the previous one. >> This one. >> Yes. >> Yeah. >> So this looks like a very complex >> Yeah. Yeah. This is a very complex this was uh in practice also it was very complex to implement. I would not right away recommend it if you do not need it. It is also costly okay because every um certif there was site car being installed if you know what sitecar is and all these pods are actually consuming a lot you will have to have a certificate validator with every microservices >> so may may I ask from delivery perspective how many devops engineers did you require for this >> not just for this for the entire setup >> um this was actually done by two person the whole PC's and everything uh if I may be very precise uh apart from me uh because I was kind of uh you know leading them there was one engineer who did the whole PC's and then uh later on he explained like because for we we did a P first and then uh two engineers implemented it and how we implemented is we codified it with the terraform all right and once that was done it was extremely easy to deploy uh deployment was not a problem >> from the certificate configuration perspect did you use the same certificate in ty and engineext >> no t used um the you know third party certificates the ones like godaddy and things because t was the also api gateway which will be uh integrating with cloud >> the internal ones were using a certificate manager internally because it would be too costly if you purchase a god certificate and install it. So we had a wild card like star domain and we actually used uh subdomains from that. So it was quite cost effective there but internally we were not using public signed certificate internally it was >> and when we use sitecar and this internal gateway normally it increases the latency because you have several layers and certific >> it was only done in production and there was a pre-pro environment which we had we kept it for audits and kind of staging purposes uh but others uh we did not have uh like these many number if I may say these many number of services were not deployed all the time. So we we had like one part but still we to test the certificate we had a separate environments. So we were still using uh like minimal number of port numbers. So yes there is a cost incurred uh that was budgeted >> and the latency was acceptable to the customer. >> No latency we increase the input output uh bandwidth within uh GCP. We asked for more input output operations. You can you can actually purchase that. >> So on >> within the cloud, right? >> Yes. Yes. >> Within the cloud there was not much latency because it was all on the same VPC. But yeah, the consumption was more. So we had to purchase more bandwidth beyond what uh GCP gives you by default with the VPC. So yes, we did have to bump up our uh you know. >> So this was all within the same VPC. So latency was not an issue as such. >> Yeah. >> Got it. Thank you. Thank you. Very interesting. >> Yeah. >> Uh this particular Okay. You talking about the all the service mesh thing everything. >> So this was on GCP. Uh we were using GKE and uh service mesh was from STO. So STO is open source. So used to use that monitoring we used uh GCP's logging right and we also had uh ELK uh elastic search that was self-managed so we had set up an Elk cluster because we needed more logs plus for the application level instrumentation like I mentioned we use dinat trace so dinat trace does an agent level like you can install dinat trace on a kubernetes agent uh kubernetes pod so we have to monitor application level bugs also >> yeah It was handled by dyn. >> No, Dino trace would generate alert. It was more logging. >> So it will it was for application monitoring. It will collect the data and stay stored there. For logging and monitoring and alert generation, it was ELK. So different kind of alerts are generated by different machine. Plus infrastructure level alerting it was GCP's uh I think it was called at that time stack driver. I don't know what it is called right now. They have changed it. They have some logging or something. They have changed it since then. >> Okay. almost this was duplicated recently. >> Yeah. Yeah. So that's why I'm saying this is older as I say right >> in the newer architecture >> I am not aware in the newer architecture what they have used I'm no longer with that project it was an old customer that's why I'm able to talk about it because I am beyond there if I was there I would not be able to present it right so um I think uh the engineext was replicated but AWS they have their own u agent released GCP also I saw somewhere they have released something um but yeah uh it was a big I think recently there has done any any other question. >> Uh zero trust controls produced minimum defects. >> How um which zero trust controls produce minimum defects >> or reduce the defects? >> I would say personally from an infrastructure side I like the blacklisting of IP addresses. I don't like allow deny. So I the first thing I do when I'm setting up infrastructure I I actually create everything on private 100% private even if there is no internet access because then I can slowly phase by phase start opening up access on internet um which I I know what I'm what I am opening up. >> Yeah because it is always good to be aware what you are signing up for rather than you know sign up for everything and then being aware. So I think that one part uh helps in minimizing the >> follow are there any controls that are difficult to operationalize? Yeah, this one is difficult like I said right mutual TLS anything on the application layer is very difficult because it impact the application performance impacts uh cost uh of deployment right and also managing right um the team here was extremely small but they were extremely good I would say savvy people so it it just had a team of 7 to 10 people over time um so uh yeah but in some organizations uh you would need more dedicated team if you can like there is a cost uh effect plus performance you will have to you will need more consumption metrics like in AI age you will you may need more tokens when you are you know because you're constant validation [snorts] >> can we take this question offite >> okay thank you >> thank you