Bringing AI to Low-Income Countries to Boost Global Health, Africa Periañez Founder at benshi.ai

[Music] africa welcome so lovely to see you very nice to see you too africa you're our first our first keynote speaker of this conference so you have the responsibility of the big opening and breaking the ice so we're looking forward to listening to you all yours i will do my best okay well thank you thank you organizers okay well thank you elena so it is really a pleasure to to be here today so so indeed i mean the last tech conference i participated uh before pandemic it was a big things so i had very very very good memories of that so hopefully this year we can see each other again so okay the subject of my of my talk is about bengiai okay so thenxia is a non-profit organization um funded by the villa melinda gates foundation as you said to to bring uh the latest ci technologies to the most unserved communities in the world so let me let me talk a bit about the about the name okay so ben gi is um it's a japanese word it's a japanese profession that doesn't exist anymore so if it's a profession that when the cinemas uh when the films didn't have a sound they were only with subtitles so this uh dimension were the people who was explaining what was happening to the film to those who didn't know how to how to to read okay so this is what we want to want to do okay so we want to provide our machine learning platform our algorithms bring those technologies to read what the data say in those uh in this sector so and also i mean most of the of the leadership and members are coming from japan working there before so it's a it's a piece of our roots so so yes as i said um our focus our vision is to to reduce um health inequalities with ai technologies so more completely more specifically is um to provide real time and just in time personalized incentives and recommendations to frontline health workers and patients so basically provide adaptive interventions and how we are going to do this let me let me put some examples and let me start with um um with some background about the usage of of mobile and health applications in lower middle income countries so [Music] the when we are talking about um about digitalization about data about machine learning in lower middle income country settings so the first word that appears is mobile okay so nobody is used for everything so for instance just one example so the the payment apps um were not invented in u.s or in china it was the first one was in kenya okay so the what is happening um in low in non-income countries about this is is big okay so there are many apps the differs from the apps that are running in high income countries so like for instance in terms of that they're able to run um without internet connection and also they adapt to latency of data okay so and then i mean because of this landscape because of these situations the number of mobile health apps is is very big it's even bigger than high-income countries so and it's used of course to to improve health so it is used to restrain the health systems and deliver online services it is used to build capacities of online health workers and also to address behavioral change and and this is what bench is trying to to solve okay so these apps generate a massive amount of data that is not used whatsoever to to improve healthcare outcomes and we believe that this can be uh can have a tremendous impact [Music] so let me put some context and some examples about the the projects we are we are currently working so and some of the of the try i mean of the global health um challenges we are trying to address or to support with technology so maternal and neoborn mortality is the greatest disparity between low and high income countries every single day more than 800 women and 40 000 babies die from causes that are fully preventable um for reasons related to maternal care and 99 of this happens in non-middle-income countries midwest saves lives okay we need to empower them so we need to to help them first they are not enough and second their training can be definitely definitely improved so so this is why i mean there are many organizations trying to address this and mobile phones i mean mobile apps is the is the communication channel um and to to to solve this and this is why we are partnering with organizations like maternity foundation to improve um the the skills to build the capacity for midwives to to help on this on this aspect so um [Music] another example [Music] wait a second okay so another example is malaria okay so for instance in 2018 there were more than forty thousand for four hundred sorry thousand debts and it's one of the greatest global health challenges so with 1.5 million molar cases since 2000 okay again there are many apps that they are trying to to help them to and to reach areas that they are very difficult to otherwise to to that healthcare workers go so be able to survive to provide some violence and also to um address and to support uh prevention guidelines so this is um this is also um there are a lot of data generated with this so normally i mean there are rapid tests to that formal area that if we make a picture and support um the if the test is positive or negative so we can have much more information about what is the situation of the if there is an outbreak of malaria and again support with recommendations and predictions when is that more likely another outbreak happens and also how to support the healthcare workers in the area and also the the the citizens to to take prevention uh measures another example that we we work is related with hiv okay so hiv is all behavioral okay or most of it is behavioral so for starting for the prevention guidelines also to adherence to the treatments to antiretrovirals once you get a positive and also how to avoid if you get a positive to to transmit the disease to other people so again some the the rapid tests their apps that make a picture and also connect with the behavioral analysis and behavioral interventions to try to improve the situation to try to improve the that the past tense pick up the test once they do and to have definitely with all these um to the healthcare systems in the in those areas sorry another one that we are currently working is with supporting pharmacists pharmacies are crucial so pharmacies are the first point of contact and sometimes they only contact with a healthcare system with of many people in the world so supporting them with them with information about which medicines are available which medicines you should start um ordering because are going to it's going to take time until the supply um arrives okay um support pharmacies to provide the right guidance to the to the patients and also to avoid the rational use of medicine is something that we are also working again through apps that connect pharmacies that support and help with them with the supply of crucial medicaments and medical um devices so for instance wgho um prepares a list of the of the of the of the drugs that they're very important that every pharmacy in the world a drug store has to assure that the essential medicaments are always um are always available for them in every area of the world is a must and also we we have partnered with the world diabetes foundation to support online health workers in low income countries to um to support to diagnose and to take early treatment with diabetes three out of four people with diabetes are now living in lower middle income countries so it's also very important that that this primary care is aware and support them to with the right information with the right recommendation with the right intervention to to to help to with the diagnosis of this disease and for this purpose we have uh we have built a data centric and behavioral machine learning platform for low and middle income countries so let me be more specific so why we say data centric okay so of course we have realized that uh one of the one of the most important points and this is i'm going to say i don't want to say anything new is the quality of the data we are talking about digital data so that the quality can be can be the best okay so this is why we have developed an sdk that uh help our partners to to track and to label the data properly for machine learning purposes okay so automatically they integrate this sdk in the code of the app so automatically it's going to track us as a satellite um with machine learning purposes and then the second step is that this information is going to fit the data governance the data pipeline awareness model governance that is available in our machine learning platform so automatically i mean they will feed the aza features the different machine learning models that are in production behavioral machine learning models and also advanced advanced experimentation like reinforcement learning so all these with one goal with the goal of empower for online health workers and to provide them with real time and just in time adaptive interventions so as we saw before and and to close the loop okay so what the kind of recommendations the kind of predictions and interventions that we provide to the friendliness of workers have different goals so the first one for instance building the midwives capacities so a well-trained midwife is able to to save two-thirds of the current debt so supporting primary care diagnosis like with what we do with wool diabetes foundation or prescribe the right medication at the right time and again be able to support with um other diseases if the medical diseases like um tuberculosis malaria or hiv [Music] this is just um some some screenshots of our platform so what we want to show you and also have one video demo included in the presentation is that um i mean our visions like of course there's a strong back end with the sdk and all the production models and so on but we have an intuitive and actionable front end that every partner has access to their data they can download if they want to do any tailor further analysis um they have full transparency about the models that are in production the accuracy the features that i introduce all all the experiments that they are running and then and the and the impact they are having on the outcome of interest so trust i mean being transparent is essential for us it means so they can have full access but the idea and why we're building these is that what we want is that every single organization in lower middle income countries have access to this battery of technology both from modeling recommendation experimentation so they're gonna start i mean of course we have them but also um these uh um allow us to scale allow us to um that the organization with uh i mean maybe one or two data analysts they can have access to those technologies that before and they never had access no no no sorry this is another another analytics because we also include analytics because what we have surfaced is also important even if it's not the core of our platform is that analyzing the past showing the results and the campaigns the effectiveness of the campaigns they performed before is also very interesting and for them so this is also included okay here i have the the video just in order to have a look about what we have built we are going to release a new version this month but it has login and well you have access to the summary of the results analytics of every partner has a access to it how many experiments has run also the data ingestion how is the the the data that is coming the models that are in production so as i mentioned before with a very strong model of burnons so every single change in the i mean if there is a new training set for running every parameter or new feature that is included is also visualized also the accuracy of the models um here we see like yesterday users at the individual level um and also the experimentation engine where we have um like clinical trials uh heavy testing we have also microrandomized trials and enforcement learning in production and also you can have access to the analysis of the impact of the different interventions or notches that that every organization have sent and also with the fed the house at the individual level so personal personalization is a key for us and i think that's all okay so okay so which we tell the data i mean which data we are we are working with okay which are the what is the information that that we track with our sdk and also we include as a external information but it's included in our in our platform so the first and the main source of information of course as i mentioned before are the logs from the apps okay so we have um a lot of friendly health users from frontline health workers sorry and also sometimes also passion that introduce information and also healthcare workers introduce passion information embedded into their apps so all these logs all these records all this information is what is track okay so this is the main source of information in terms of patient um health profiles that are introduced by the healthcare workers uh we also analyzed the trajectories of health and disease and also how the actions or different health workers or even the same health workers can have two different pathways so it's not the same so we personalize that at the two levels we can say and also contextual information where i refer with this we need to to put context to the recommendations we give so we also uh partner with public health and government governments governmental institutions to have access to democratic demographic information environmental information cultural religion very related as well with nutrition that can derive to different kind of diseases as well or can trigger some of them climates if it's raining or um or dry season it's also fundamental if we want to recommend for instance that the person is uh is sent to to a clinic and also a epidemiological status okay if there is an outbreak of kobet of malaria is fundamental that all this information is included as obviously the the recommendations will will change so as i mentioned okay so um our goal is to provide useful and actionable information to first understand past behavior okay understand how providing to our partners with this information about what happened in the past what decisions were were right and and which ones were not so successful and also predict future outcomes so who is more likely that get a certification in the online learning app who is more likely that stop using the app and the condition is going to be broken who is more likely that has complication in the in the current decision you need to increase the the the amount of business the frequency of the visits and all this information is to take action and match behavior so so once we know for instance that is um is is likely that uh i mean a midway is predicted that it's not going to get a certification but it's going to be close to get it so how we can motivate her to to be able to to achieve the goal so this is the the core of our work and as i mentioned before as well i mean personalization is fundamental for us so we we of course personalize at the level of user okay so every behavior that this person does and and and all how the contextual information affects to to this particular user and then moving from the user behavior to collective behavior so we use it's much easier to to be able to to to understand the different profiles also we are talking about collective behavior and also when um something that we do as well is that even if we focus on digital information we also perform um users interviews on one site okay so it is important as well that um that we are able to determine different profiles so at the end i mean we we want to check if this uh this analysis is correct with uh personalized interviews and also surveys um also a personalized passion level as i mentioned before the actions that the user does um can have a strong impact different impact on different patterns and also personalized at the level of pharmacy clinic of point of care okay so it's not the same a pharmacy of a clinic that is in a rural area in a remote area that is if even if it's low income in settings in a in a city so the kind of recommendations we also perform in terms of supply chain or availability of drugs or also the the the patients that go to that to that clinic or pharmacy are completely different so the recommendations also adapt to to this level of personalization let me explain a bit how the the work uh cycle okay of the of our platform works the first point is the is the sdk so as i mentioned so there's the case integrated into the code of our partner's path okay so this allowed to communicate the first one is like a real-time thing of the of the logs and also way back okay so not only like the data is coming the data's going so we were also able to send interventions not just not just and track the the impact of that so okay so with sdk we receive all the information about the what is what is the actions and how is the interaction of the user with the app the second one is that automatically is going to be labeled and the data pipeline differentiating between metrics kpis trades and features okay matrix is just individual time series behavior kpis segregated traits are different characteristics of different behaviors of users and the features is what is meant to be included as um as a feature of the machine learning models and then this information that is organized and classified in a very rigorous manner is going to fit the the the model management okay and here is where we have we have a product that organize and keep track of every single change that uh i mean that we do in the models that they are in product that we put in production if we are running the model with a longer historical data this is also been um uh controlled okay or if we changing a parameter of course if we change the algorithm both from algorithm inside and from data side this is going to be uh tracked into the model management product and we focus on three main kind of models three blocks we can say the first one is the user behavioral predictions this means that we are going to we are going to predict uh at individual level who is more likely to get a certificate a certification or not who is more likely that they stop using the app who is more likely that is going to have a complication with uh with the pregnancy or with the the current disease so these are the kind of use of behavioral predictions that uh that we perform for every single user and then for instance once we know um that for instance uh uh a person is going to stop using the app or a person is going to to ask for a particular um drug in the e-commerce app so we can recommend what additionally um we can i mean can be content can be products we we can we can provide um to the to the user to to take a better to improve our healthcare outcome so in the case for instance that what is the what is the right information we need to send them i mean in order that that's a better diagnosis for instance and then forecasting okay forecasting is also a very important piece first to to forecast the contestant information conditional information sometimes is not updated okay so it's difficult to keep track of all these demographic environmental um information so we also perform projections and forecasts to to have more realistic information for the contextual data and also in terms of forecasting of course we include um trajectories of health and disease but also the supply chain supply chain it's important that with the limited resources that they have so it's very important that they don't spend too much time making plans of purchases and so on in a very dark um and hidden uh supply chain system that sometimes is very difficult for them what is going to be available and what is not so helping in optimizing the the supply chain helping in uh in keep things easy and send reminders about certain materials or medicaments that needs to be purchased in advance is also very important so with all this information okay and and again i mean our goal is to to nudge behavior to take action it's where we go to the natchez service in the nazi service we choose to whom we want to perform an action and the action itself which kind of um of of recommendation if it's machine learning base or just message if it's a post notification we want to perform to to the different people for instance if we imagine that we want to send some reminders of online head workers that work in uh in nigeria okay but you want to focus on the lagos area only okay so so for instance this is a place where you can choose even a subset of the results um that they are coming from the from the from the machine learning models so and then the the next step is the experimentation engine okay xp engine experimentation is permutation experimentation so this is this is extremely important so it's the experimentation is the only is the only way to to to casualty and is fundamental to to understand what is the impact of our actions are having to the to the different end users okay so this is why in this um in this product we provide um classical techniques but also micronized trials and reinforcement learning in a in a basic and more advanced ways to help to give recommendations on the fly um first experimentation then when we are sure we can do full adoption and everything both experiment experiments and full adoptions are done through the sdk sorry we also do research in uh in-house okay and and these are the main pieces that the main the main research areas that we focus inference is of course one of them as i mentioned before behavioral prediction like working with the with the most advanced behavioral prediction algorithms working with state of the art and just published algorithms trying to transform these theoretical approaches and put them in production but also in variations of the main algorithms to to be able to to adapt to to our needs or challenges forecasting also as i mentioned reinforcement learning of course and also synthetic data generation so it's very important that we test everything carefully before breeding production with our partners that is why we try to simulate synthetic um with synthetic data actual user's behavior um we have i mean we only have been alive for one year okay so but we are happy that we have two three three main publications um uh i mean i mean three two papers as a default publication sorry so the first one was in kdd and we just go to two purposes of center for new rips in the in the worst of the first one is in the andrew and g of data centric ai and the second one is the public health worship in eurips and we were also very happy that that the one that we that we sent to kdd and it was award with the best reward in the healthcare workshop and we are doing in-house research but we also have key collaborations with um with uh with the best teams in the world in terms of what concerns us so we work with harvard university with the statistical enforcement learning lab but there is a specific lab that focus on on applying reinforcement learning for mhealth applications we work as well with the university of california santa claus with them with game design elements for healthcare okay so we are coming from the from the video game industry and uh we also feel that uh we also know that um video games in terms of nothing behavior in terms of motivating they are unique so also bringing those those elements to the to the healthcare world can be can be extremely impactful and the last one is the university of tokyo that it was just recently a signed agreement so the focus of this is um is collaborate with fundamental research on law enforcement learning so at the end what we are doing is adapting algorithms they are working extremely well in in high income countries but but what we want is that also develop our own algorithms that work specifically for our settings for low middle-income countries so that is what we are doing with them with the with the lab of machine learning of the university of tokyo this is one of the biggest if not i mean i think the biggest in in japan in terms of machine learning research and before i continue i want to just stop one second and talk about the team okay because because we are working restless and and they deserve definitely acknowledge um for for the work that we are doing so it's a i mean part of the leadership uh we're working together with some of them since 2000 2015 and back in japan and they moved we moved together here to spain and well i mean i don't want to say anything new that all of you know that for data science teams diversity is fundamental and for us that we work in in in lincoln country settings is is definitely a must okay we if if if we don't have a truly diverse team so we have people from china from singapore from iran from pakistan from haiti from nigeria from korea and along etc and we are really working very hard on this and this is not trivial okay so because it requires a lot of effort from immigration's point of view and but definitely uh the results are better and and it's worth it okay so [Music] i want to trust me this also for everyone and [Music] also okay so now um i move to the piece of the collaboration we have just a case study with the maternity foundation maternity foundation is a non-profit organization that is the headquarters in denmark but the focus on the work is mainly focused on africa and also india this is one of the samples that i'm showing here so the focus of of maturity foundation is to provide with digital tools and with apps that increase the capacities of um of midwives okay so don't power them so so basically it's an online learning tool okay so that they try to cover um the i mean the most important skills that they need to to reinforce or to to to be fully updated so so for instance in the first figure we see the number of uh safe delivery apps there is the app that has been developed by maternity foundation so safe delivery users in india okay so here we see a distribution and then for instance we see like the district level uh poverty in a different in a different uh dimension so we can also see that the number of users and the number of um of i mean and then and the level of property in india also correlates and also when we have a loop not only for the user that have download the app but also when we we see the level of engagement that those users have with them with a safe delivery app it also correlates with the areas that they're poorer in in india so some some of the some of the analysis okay just to put a couple of examples of the of the work that we are doing with maternity foundation so one of the goals that um that we work together is to increase certification okay to to get a level of knowledge and exam through a series of personalized and contextually tailored interventions okay so so this the results i'm showing on the right are predictions on learning progress among safe deliberate users in ethiopia okay so so here we see different professions okay not only um our midwives or other skill birth attendants that that use the app and here we can see so that is there is a a change of behavior uh below and above the level five of them okay so and um the ones that i mean once you pass the level five is much more likely that you're closer to get the certification and we can see the different distributions of the different of the different professions and we can see that the students they are not so i mean so keen to continue using it but what is the big need this if we midwives but for instance if we go to the next slide so also we can see the different distributions of progression and at the end getting the certification so again in the level five we see that they are the ones who are less likely to progress into the into the learning app and uh and and this at the end this information will tell us these different levels of of course personal life interventions we need to perform to to in order that they continue using the app and continue learning okay and um and then with having the the right level of engagement that this is the goal it's not like uh with other apps that the more they use it the better okay you know the is something that supports the learning curve of the of the skill birth attendance so [Music] so for instance the more you use the uh in case if if you spend that this three hours using the app you is predicted that you go below the the level five so be able to provide with personalized content not only to pass the certification for those that get that they're going to be closed and and it's very likely that with some information that they get it but also to be able to to send information that engagement contents modules that that that they enjoy in order that they can also learn the ones that they enjoy less okay so be able to to recommend personalized for every person for every circumstance and if if and it can be much more aggressive if it's very likely that the person stopped using the app that is for instance is in an area that is more likely that naturally happens or is very close together and and also to finish okay so the last piece that uh that i wanted to talk is about reinforcement learning okay because enforcement learning is a fundamental piece in in our platform and uh and something that we really want to bring uh the state of the art um to to low and middle income countries so enforcement learning consists on an agent that learns through interaction with environment what is the best action to maximize reward for a given state okay so in our case um the agent is going to be basically a platform where the algorithm is in production the action that we perform is intervention okay with a specific um healthcare goal so the state sorry i forgot so the state can be any information of any environment representation can be any any information about the user behavior or the the information about where this person lives demographic status etc and the the environment is also the the behavior of the user the passing the contest and the reward is the user behavior and at the end passion outcomes okay so depending on if imagine if if the goal is that the healthcare worker revised some information before doing diagnosis or other includes some additional tests to the passengers okay so if at the end this person did it this would be a reward it did not it's not okay it was not successful that that intervention okay in this uh in this slide i included um a slot machine picture okay because i i wanted to introduce the multi-arm bandits so um slow machines are sometimes referred as uh i mean i'm not i know most of you know as one are bandit okay so why because the the old ones had this um this arm okay that you can pull to start playing and also bandits and this is it where this name is coming is because they typically reap your money okay so and the enforcement learning can can be represented i mean the simplest problem okay is uh it's called uh multi-r bandits okay because it's uh it's equivalent to having a series of slow machines and you can decide at this time which arm you should pull to try and maximize your work or at least to to to to minimize the losses so this is because i wanted to talk about the multi-r bandits so this is the simplest version of the reinforcement learning problem where the actions are africa sorry sorry for interrupting africa but we ran out of time uh i know you started a bit later so we have to start finishing if you if you may if you can that'd be so kind thank you okay so basically um [Music] uh let me just summarize quickly okay so um well we use uh starting from multi-urban days we move to contextual boundaries where information is online so we have also information about the state and we use enforcement learning okay for first personalized intervention so with other experiment type of advanced experimentation is not possible to take into account online information again and that the systems adapt to learn even if the best action changes and and also just put some context about the different kind of experimentations that we also have in production to so like every testing randomized um control trials mrt's where everyone can be in a and b um for different in different days okay to study i mean how they how the intervention can um can uh how intervention can change depending on the hour that they they sent and moving to contextual bandits okay so basically multi is a smart version of randomized controlled trials or mrts and contests even a smarter version of reinforcement of a randomized control triangular or mrt's okay and we are working of course with synthetic data research and also the direction we are going is towards collaborative interactive recommenders where not only based on actions also focus on sequence of actions um there's a summary so our goal is to empower with information based on data from like health workers in non-middle-income countries reformed learning is extremely powerful and putting together information about analyzing past behavior predicting future behavior africa africa sorry for interrupting again i suggest our our viewers take a picture of this summary so they will remember you don't have you have been stalking and you're thirsty take a picture because we have a few questions uh i i we don't have time africa to i know you started a bit late and i apologize for that we have a big a few questions but just a quick one before we say goodbye to you first of all thank you so much congratulations on the amazing job you've done in so little time and one of the questions i'm asking is just the one there's a yes or no answer whether you're working on any you mentioned some of the global health challenges hiv malaria diabetes obviously birth rate mortality at birth what about kobe 19 are you working or will you be working on any of the in this yes or no and a very quick tweet answer yes we will be working as well excellent so we have to stay tuned then okay africa i have to thank you so much for the fascinating talk amazing job congratulations arigato gozaimasu and we'll see you very soon africa periano [Music] you