Submind YouTube summaries
Thumbnail for Discover anywhere, move only what matters - inside InMobi's DataBridge

Discover anywhere, move only what matters - inside InMobi's DataBridge

Watch on YouTube

Video summary

The video introduces InMobi's DataBridge, a practical implementation of the data mesh philosophy designed to solve critical challenges in cross-domain data sharing within an adtech environment. The core problem addressed is the inefficiency and governance risks associated with traditional centralized data lakes or ad-hoc copying methods; centralization often dilutes ownership and creates cost accountability issues, while manual copying leads to fragmented local copies that are difficult to manage and prone to security breaches involving sensitive PII data. To overcome this deadlock where valuable exploration happens before materialization can be approved, the team developed a system prioritizing global discoverability without moving underlying data until a specific business use case justifies it. This approach ensures that engineering resources are not wasted on exploring potentially useless datasets by keeping them in their original silos initially and only replicating high-value data after rigorous approval processes involving legal and compliance teams. The technical architecture relies heavily on Apache Iceberg tables to create a unified metadata layer where the actual data remains stationary within its respective business unit's storage, such as Google Cloud Storage or Databricks Unity Catalogs. Instead of physically migrating petabytes of raw data, DataBridge registers source catalogs into a central Polaris catalog and uses pointers to reference specific snapshots of that data. When queries are executed via engines like Trino or Spark, the system validates access permissions through an OPA agent before issuing short-lived tokens to query the original GCS locations directly. This federated querying model allows users to perform complex joins and aggregations across different domains seamlessly; only when a consumer team identifies a valuable intersection of data do they initiate a materialization request, triggering an automated process that creates a local copy while maintaining strict governance controls over who can access what. To ensure the system remains useful despite being distributed, DataBridge employs hourly sync jobs to update metadata pointers and handles schema evolution by tracking version changes in Iceberg tables. The platform supports various materialization patterns including simple projections of specific columns, aggregations across partition keys, and join operations between different datasets. A significant learning from this implementation is the acceptance of eventual consistency rather than real-time synchronization for all data movements to avoid excessive latency; while current updates happen on a scheduled basis with plans for future real-time capabilities, the system effectively manages pointer drift by refreshing copies only when necessary based on snapshot differences at the partition level. Additionally, the team standardized column naming conventions across disparate business units using glossaries and conformed views to ensure that explorers see a uniform data structure regardless of where the source tables originated. Ultimately, DataBridge demonstrates how organizations can balance autonomy with collaboration by enabling discovery without duplication until value is proven. The solution has successfully onboarded nearly 100 datasets from various sources, facilitating over 30 materializations totaling approximately three petabytes of processed data while maintaining robust audit trails and immutable definitions. By shifting the paradigm to "explore first, copy later," InMobi's team reduced unnecessary engineering effort and prevented the proliferation of unmanaged local copies that often lead to compliance risks. The key takeaway is that metadata management should be centralized for visibility and governance, but actual data movement must remain consumer-driven and justified by concrete business needs, creating a scalable framework where different teams can leverage each other's intelligence without compromising security or operational efficiency.
Read the full video transcript
Hey. Hi. Um, so this is uh a talk that that we want to uh start with is in like they're like uh in uh currently in my company there are like different business units and all of them have a different cataloges. So this talk is mostly uh uh explaining how do we govern data sharing across different uh lakehouses? How do we make sure that that the data that that has been shared how do you have governance around it? So this is a a kind kind of a talk of uh kind of building a practical data mesh uh for cross domain data sharing and we'll want to introduce a kind of a principle where you want to have a discoverability across all the data sets uh but then have certain use cases to just to metalize locally. I'm uh Shom Gupta I'm currently a staff engineer at InMobi uh team. So, Inmovi is mostly an adtech uh space company and this is from the central platform team. Uh so, I'll just go through the problem and then give you the brief overview of what what we try to build and how we want to uh make this learning available for you. Uh this is mostly an uh a talk where you want to go through the architecture of how that we build it and what are the constraints and lessons that that we u had and what what did we do about it. So this is the main business problem uh where you'll have multiple business units. They are residing in their own silos. Uh they have their own catalog. Uh they have their own data formats. Uh and each each of them are handled independently across them. U autonomy is kind kind of a very good where every data sets resides into single islands but then data sharing never happens. You are not uh you don't know how to access data across the domains. uh but the value comes in when the data where you can get the data of other view do some interactions around it and then create value out of it. So at least from the ad tech space u so one of the examples is where you can get an audience overlap. So let's say if I get a user doing some activity in in a certain app if I can get that information into my ad serving platform where you can target that user again is something that uh will be really useful for us. So that is where uh your cross domain identifying the intersection becomes very critical for us. So how do we get that uh so one major way uh what people or company would have done is like centralize everything. You have data across the BUS um you'll have like data moving in um push to a central data lake. So that is like a data copy that that you'll be doing from one BU. You copy that data completely into a central warehouse. This will uh dilute your owner ownership and then there will be a lot of issues on how do you want to govern those things up? Who pays for it? Uh so if you are moving the data across to a central place um who pays and how how do you get a cost accountability to that particular system is kind of an an issue here. Other problem or or like an other solution which teams would have tried is uh doing an ad hoc copy. What what what it's do is like uh you get uh the data of certain BUS and then request ad hocly whether you want to do that. What you do is like there like multiple teams uh what you do is request certain copies get some approvals get a local copy into your system and then again you want another data you request another copy. So what happens end up creating is that lot of local copies are created and then uh what ends up happening is like it becomes a struggle to actually manage and identify who copies the data how that data is managed. Uh what ends up happening a lot of PII data uh is present in these uh databases and and it gets moved across laptops and people can freely um check all these things. So the copy diverges uh the data source. No, nobody knows how how that thing happens. So that is a big problem here with ad hoc copies and uh the biggest uh so this is kind of a deadlock or like the heart of our talk where uh you can't govern what you haven't explored. Uh so let's say if the data is useful, you want to know whether that data is useful or not. What you try to do is explore that data. But once you try to explore that data, that data of another view is not available. So what you do is like um try to request a copy uh from another BU to your BU. So what you do is like uh get the copy uh do all the governance reviews and approvals and then build pipelines to actually copy the data from another BU to your BU and then what what ended up happening is that the moment you uh like lot of engineering effort is wasted in actually copying the data and then finally after weeks of effort you're trying to explore the data which may or may not be useful. So that is where the loop comes in. Uh your old model asks you to pay for the cost of exploration before making a business decision and everything follows. So that that's the main problem that we want to solve. Uh that is where we have built this kind of a system where um onto a philosophy uh where uh you want to have a global discovery. uh you have your data across all the views but make that available to be explored across all the systems and once you've identified that this is a particular business use case uh and you are sure that this makes sense then only you materialize it u so so until unless a use case or or a or a majority of uh business is valid there's no need to actually uh do a mutilization um this is kind of a a funnel you You can think of it as like uh every data sets that is available only a handful of them will be useful for exploration if you have that kind of a data and and uh once you explore that data across the Bus and then there'll be like a small set of data which justifies uh enough value to actually go ahead and request a copy and your governance and your all the approvals of how do you want to get those accesses should reside on the last layer where you actually want to materialize and copy that data into the view itself. So that's the core idea that we want to uh pursue this uh system around. U you want to have kind of a data sets to be explored and uh be available for exploration. uh do that overlap analysis um with your data and what whatever the data is present in a specific view do that overlap analysis and then only if there is a certain business use case and you find a certain data set which is useful for you then only do a metalization so um so before going into the architecture I'll just uh start briefly on uh what is actually a table how does it do uh this is mostly an uh a standard how the cataloges works uh into this kind of a system. Uh there will be like a tables um and then corresponding to that table will be a pointer to a metadata where the metadata saves all the uh schema partition specs and all the related details of where the actual data resides and the data resides in different location. So data can reside into a completely different location itself or it can be into a same GCS path itself. So that's how an actually a table in an iceberg or a standard catalog works. U so when you're doing a write um it just creates a new snapshot. So let's say there's a catalog pointer to a table it may respond to a particular snapshot there will be like a four or five uh park files in into a bucket. So let's say if you add a new data or a new partition path, it creates a new meta uh snapshot for that particular system and then what happens like now your catalog is pointing to an older data. Uh what you just do is like point to the latest snapshot. So that's how uh the table and we want to use this kind of a schema to build our uh system. Now coming to the main architecture of uh what what we have in our system is um this where your data never moves. Um what what we try to build is kind of a data mesh platform where uh your data can reside into the BU's uh so data does not move. So all of your BU's data uh in their own uh certain cataloges it can be either unity or it can be a Polaris itself. What we do is like we register all these cataloges into our central Polaris. uh so this is like a unified metadata catalog. What we do is essentially create tables using the references of metadata present into the uh source catalog. So essentially it is uh putting a metadata layer on top of your u actual catalog. So this is where your uh all the sources u the data gets uh pulled into a central unified catalog. So this is um and then we build a kind of a control plane which is kind of a a back end to actually manage all these onboarding utilization governance all the audits corresponding to this particular system that that we have built and then uh you provide kind of a a querying engine. So so you use train overx to actually go ahead and query uh our central catalog and which internally is useful for exploration as well. So that is where you go ahead and explore the data and then we have uh kind of a a UI for uh all all this. Uh so we used open metadata which provides kind of a way to actually use uh or visually see all those tables that have been onboarded into our central catalog and then um all the metalization and all uh the that we have built kind of a custom um filters where you can go ahead and see that data itself. So going into the detail on um so this is like what the architecture of our system where data resides into your in your own specific views. Data never moves and then we have kind of a metadata layer which actually points that to a central or central system owns the metadata and then you can go ahead and search all the data. So the query spans across the um views and then you can go ahead and make a decision whether you want to go ahead and mutilize it. So yeah uh now I'll just go through the details of each and every steps how the onboarding works. How do we uh make sure that the data is updated? So first way first step is like how do we onboard the metadata? So it is just a registration flow where you first register the actual catalog. the source cataloges are registered then you can just onboard the table. So the onboarding is like uh get getting the source table and then pushing that metadata or like creating a new table which is pointing the metadata of a source view. So it essentially is like a pointer to a metadata in the source table and creating a new uh table on the uh central polaris side. So once that table is created so so we'll have like a similar table uh structure created here but it points to the metadata of the catalog or the source catalog and then once it's uh onboarded or like enabled into the central polaris we want that to be discoverable across the systems. Uh so that is where the metadata moves but uh but the data remains the same. Uh so we have like a three onboarding steps. uh so so um lot of our views are using datab bricks is unity catalog and some of them are using an existing iceberg as well so we kind of support onboarding all of them into our catalog so for datab bricks uh unity we use um kind of a uniform enabled so what what it does is like creates a metadata of uh the tables correspondingly in a format which iceberg can understand so iceberg adapters are simple for plain delta tables uh we use some something called as an X table which creates the metadata uh similar to a format which iceberg can understand. So once the onboarding is done with uh all of this tables have kind of a metadata compatible to an iceberg format and then that has been onboarded to the central catalog. So that is where your uh once the registration happens uh your tables are actually are present into your central catalog and then um the next step is like once the onboarding has been done how do we make sure that your central catalog or central uh actually stays updated. So since uh what can happen is like producers uh what we have built is like producers can keep on uh producing new data. Um what we have built is kind of a sync service. So what it does is like tries to see what is the latest snapshot that is present onto the source side and try to sync that into our central polaris. So currently we have kind of an hourly job. So what it does is like tries to see what's the latest snapshot or the latest metadata on the source table try to get that snapshot or metadata into our uh central polaris itself. So what it does is like essentially it is moving the pointers uh from the previous snapshot to the latest snapshot. So it's as you can see this is just a a movement of pointers from um the previous snapshot to an higher snapshot and then you can have all all this uh can be explored or or via trino on databick where you're and then the same things can be done on the open meta as well where where you can see the latest snapshot or latest schema of the tables that have been onboarded. So currently this is an hourly sync where we try to see if there has been latest changes and we want to sync that uh into a central vol. Now since the date now the uh what we have done till now is like uh moving the data or the metadata into a central catalog and making that up to date. Now how do we want to query it? So when you do a query uh what what we use is kind of a SQL query and that query can be uh fired via a stino and a spark. So we have kind of a two offerings where um most of the queries go via trino and then let's say they are like uh some of the data scientists and uh we are which are more familiar to datab bricks for that we use spark. So then what it does is like it queries uh the central polaris to get the metadata and uh central polaris gives you kind of an uh scoped access. So we use something called a vended token. So it creates a a short-lived token for the tables or or the GCS location which you're trying to access and then it uh runs the queries or like once it gets the metadata of where that actual location is and a correspondingly a token then it fires those queries across the the BU's data sets um and then merges those uh cross joins or across domain searches uh merges the results and responds back to the UI. So this is how the u uh the system that we have built. Uh it's like only the results are actually moving across the network not the underlying data. So yeah and this is how the actual query runs uh where when you run a query uh it tries to see uh whether you are the correspondingly access to actually query that system. So that is where your engines where where we have set up some some rules on how who and what can access these kind of a data and we have and then what what we do is like when that queries is done on the polaris side then there are like certain arbbacks that we have set up so that like uh it creates uh let's say checks the permissions uh so so that person querying the data actually has the access to explore that data sets itself and then once Polaris confirms it create it lends a a token uh for you to access that path itself. So once that token is there you uh then query engine goes directly to the GCS bucket and tries to access that data directly. So yeah once the uh so we have created kind of a unified uh analytics platform where uh there are like certain systems um that is built on top of uh polaris catalog itself where you you have some sort of a krino for our ad hoc queries database for let's say uh mostly for ML platforms and ML engineers to actually go ahead and see that data superet is mostly for some some reporting and vi dashboards that you want to uh show up Open metadata is where we want to discover the it's mostly a discovery portal where you want to discover all the data set schemas and their relationships lineages and and all the stuff and all of them converge or let's say converge to a central uh Polaris catalog. Uh now coming to the uh main part where you want to mutilize it. Uh so milization is where um the everything comes together where you want to have a a copy of that data into your uh DU's network. So that is where you want to have all the query running and explore all the data sets. Once you have made sure that this particular use case is actually um useful for me and I can productionize it, I can derive some business unit out of it. Then you go ahead and requestization. So this is initiated by a a consumer where teams asks for a local copy then it goes through the approval flows where you have an infosc and a legal approval where whether you are uh you could go ahead and actually uh make a copy of that. So all the governance layer sits here uh you have kind of a consent gates whether since you are copying the data from consumer to kind of a producer. So we'll try to get approvals uh from the consumer and producer side. um once the once all those approvals are there then only we go ahead and u complete the mizilization request. So then we uh had kind of an automation kicks in where we try to create a a copy and we try to keep that copy up to date as in when the source data gets updated. So we support kind of a three use cases here uh currently. Um so first is mostly a projection where you can do select uh some filters uh some some select columns and all so that uh the exact columns gets transferred to the producer from producer to the consumer. Other one is aggregation where we support some sort of a group by and some summation aggregations across the partition key. So let's say you want to have some across a certain partition key of of our table. So uh that that is another pattern that that we support here. Another pattern is actually an align join. So what what you can do is like you can select two particular tables and then have kind of a joint query on top of that and utilize that joint result itself. So this is the kind of a patterns that we try to support here. Uh and then coming to the mutilization. Um so what we do initially is like just the get the latest dump uh from the table and then do a a complete copy into the B used network. So the first metalization is actually a complete refresh uh where we check each and every partitions and uh we just dump that information into the BU's catalog and then the next metalization when it runs it now needs to know what changed. Um so what we want to do is like kind of an incrementally refresh that. Um so what we kind kind of a use is kind of a no uh we save the last state that we metized. So when we did the first mutilization what we do is like we try to uh watermark uh that this is the snapshot of source um table that that I tried to uh mutilize and then when the next mutilization starts in what it tries to check is like hey the last mutilization run the source BU was at snapshot 42 and then what's the current snapshot so let's say the snapshot is 548 now what it has to do is now identify the difference between um this particular snapshot and the latest snapshot. So what we try to do is like uh kind of a identify uh the partitions that that have changed. Uh so iceberg provides a capability of kind of a logical uh it exposes kind of a logical partition table where you can identify that these are the partitions that have changed across the partitions. Uh so in that part uh partition table it shows the last time snap and the last uh snapshot version that updated it. So using this mechanisms we are able to identify that uh from this snapshot to this uh through the latest snapshot what all partitions have changed. Uh for us um so so it mostly identifies at a partition level what changes have been done. So if there has been a certain a single row that that has been changed uh we'll go ahead end up uh updating the complete partition. So this is kind of a trade-off that we have taken um to identify the changes uh in a faster way. Uh so yeah that's how we identify the change in the partition and um the next thing is on the schema evolution uh where if there has been certain changes into your source uh catalog schema how do we make sure that the data copy that gets uh produced into the uh consumer how do they get updated so we uh we use we again use iceberg's native uh schema version support. So iceberg actually saves a lot of metadata of the schemas that that that it has evolved to. So it's uh so every new update that that that you do it creates a new schema version and all the schema versions are available. So what we do is like we try to save the the last schema version that it was mutilized to and then when the new malization kicks in it tries to identify what's the schema version of the latest uh metadata and then tries to identify the div. So let's say there have like new columns added if there have been new columns dropped. Uh we can then apply certain configs what to do if the there has been certain changes into the uh schema itself. So let's say there are like new columns added uh if those were not used in uh metalization then it doesn't make sense. If the columns had been dropped u then you want to see whether those columns are actually used into your metalized table. blend then you want to have some sort of a config whether you want to drop it, stop it or like move forward. So that is what our uh current system does. Um we have currently exposed it as a configuration file where you can uh try to stop the tag uh whether you want to um stop if there are has been changes in the schema pause or like notify or there's some config of auto evolve as well. I'll just move on to the uh major learnings that that we had uh with this kind of a solution. Uh so the first learning is like u we are moving the metadata across like there will be some changes uh there like pointer drift uh uniform lag that could happen. So since we are we avoided moving the data we have created a kind of a consistency problem. So what we have made sure is like we took a conscious call to uh make it an eventual consistence system. So if there are any delays um so the next copy or the next mutilization will actually kick in and it will get that latest data updated. Another is like on the query uh because what happens like these queries run across the BUS um um since we have avoided uh moving it to a central place. Now your queries are running across the BU itself. Um so that is where you want to profile how your query layer is performing. How can we do some sort of a caching around it? Whether that can be optimized. Um and we do a kind of a token pending where we try to give you a a token uh where a shortlived token. So we try to do some optimization caching those tokens als so so that like uh so so that the queries can be run faster. Another big problem of like let's say having a a shared uh system is like um lot of BU's data that gets onboarded here may have a certain different uh variable names um they can have the same kind of for example let's say user identifier same identifier can have different values or let's say some views can use user ID some can use user ID we kind so when we are onboarding that data into a central platform we want to have kind of a uniformed view or or like a uniformed uh column naming conventions across the systems. So what we build is kind of a kind of a glossery and a conformed view. So when we onboard a table, we try to see if there's actually a glossery which maps this particular column to an actual canonical uh column name and then when it's onboarded it creates a view out out of it. So that uh and then the searches are on the view not on the actual table. So that like all the people actually searching the uh or exploring the data have a uniform column or uniform can canonical structure around it. So that's what uh our database platform gives us today. It provides a a federated discovery across every domain governilization uh with approval flow um all delta and iceberg under a unified catalog. Um and we have kind of a good amount of auditability immutable definitions uh and metalization actually driven by a consumer uh platform. These are mostly the numbers. Uh we have like currently almost like 100 data sets have been um available to discover across our platform. uh 30 plus mutilization around like we have mutilized almost like 300 terabytes of almost like three pabytes of data almost like 12% of mutilizations that we have done. The key takeaway of this is mostly do have kind of an uh platform where you want to explore first. Um have consumer decide where when do you want to have a a a copy of the data not the reverse uh metadata is actually an harder part uh have kind of an immutability and audit across the the teams. So yeah um thank you for everyone. Um so this particular team uh the deep was kind of our main lead here. Uh I am Shivam. I was mostly looking into the malization stuff. Uh we have Pushpendra and uh Kulib who were looking into the sink and uh federation. Uh if you can join Thank you. Thank you. Thank you. room questions. So you mentioned when you query it centrally right so there are certain kind of projections and joins and all that you support. So can you explain a bit like what layer does that execute? Is it the TRO or the spark which is doing that or you have some custom query layer on top? Yeah, it's it's mostly TRO and Spark which are actually running the queries. So what happens like TRO queries uh so first checks the catalog gets the credentials or like a the location of those files. So let's say they are the locations in in a certain GCS it gets that location from the uh central catalog and then directly queries the uh GCS or path to get get the information. So mostly it is uh Trino and Spark doing doing the actual query >> but in that case all the operations and the dialect that is the operations that the dialect of Trino and Spark will support should be supported right. So >> yeah yeah so we try to make sure that like inter operability across the database are handled into the system itself. >> Okay. >> And the FGAC is managed what you have a ranger setup or how do you where do you define the user access permissions and policies? Yeah. So that is like a a big problem for us. U so initially what we did is like kind of a have some sort of an arbback uh defined into the trino and a separate uh system for a database. So what what we had is like two set of arbbacks but then um um what what we ended up doing is kind of a use an OPAI agent uh which will be kind of a a central uh thing where you define all the accesses uh correspondingly to a user and correspondingly let's say a certain user has certain access to a certain view with a certain data sets. So all that accesses and arbback is currently now defined into a single OPA agent and all Trino and uh Espark actually uses or queries that to get that information. you know you can still write a custom plug-in to manage this but how do you do this in spark >> so for spark what we have done is like we have created a dedicated clusters for that so let's say um what we can have is like uh different arbbacks created for different clusters so we'll have a certain group of users tagged to a certain cluster itself >> basically role specific clusters So >> you may have covered this in the beginning which I missed but the question is this kind of architecture what use cases are this is most suitable for >> so >> scenarios yeah so I'll give you one one of example where um we have certain uh let's say you want to target a particular user that user is trying to um uh let's say buy some uh things on online. So what you would have is like we have certain apps uh there's different view uh which can uh which sees that this particular system or this particular user is trying to view a certain uh event. Let's say he's trying to uh he's seeing um the uh the he's seeing the socks or or let's say he's seeing certain characters or he's browsing the e-commerce site. So what we can get is if we get those event stream flowing in into an ad serving platform then we can target that user uh knowing that this particular user is actually seeing or doing some activity in in a certain other view I can use that intelligence and target into my system itself. So this is one of the use cases that we have in adtech. There can be multiple use cases where you want to have some sort of an intelligence coming in from a another view and use that intelligence to actually derive business output output for you. Does that make sense? >> So is it like creating different data representation based on local needs? >> Yeah. Yeah. Okay. Thank you. Uh I think we have maybe it's time for one more question here. I'll just continue to the previous question. Uh would it not add to the latency because we are doing late materialization and schema evolution is also no bit delayed. So for the add tech uh would it not impact the latency both for inference? >> Yeah. So uh currently our solution is mostly uh an offline process where you have a scheduled job running at a certain frequency. So this is the current state. So we are building on uh the next way where you can have a real-time malization itself. But that that is actually in work in progress. Uh so that's an issue with the current system. It is an eventual consistenc uh in this kind of a system. So yeah, next logical progression is like can we make that real time but yeah that that is a work in progress for now. >> Okay.