September 2026 Developer Meetup: App Competition Finalists: Part II & AI Forum Kick-Off

All right, welcome everyone. Uh, welcome to the September edition of the developer meetup. So, but in this edition we're going to have uh quite a few presentations. It's the first time also we're doing it 1 hour and a half, so it's a little bit longer because we're um experimenting and trying out the AI forum. So, we know that AI is a big part of most of people's work right now. and we would like to also have some space to discuss how we're actually using it in development. So, uh the second part of the meetup will be about that. Before that, uh we are going to have uh two presentations from our app finalists uh from app competition 2026. So we will hear from the creators of Orunit sync as well as the creator of the magic glasses chew are going to share how they actually came up with the ideas and the nitty-gritty behind the app right and what kind of challenges they may have faced uh while doing that and creating the apps and then in the second person so AI forum uh Ronda Moore from his Rwanda is going to share a little bit more about locally hosted AI model to handle DHS2 data. So we hope the meetup is really interesting for you and um feel free to ask questions whether just writing in the chat or raising your hand. Uh and um then the first will be the or unit sync and the team from his Sri Lanka is going to present. So the stage is yours. Let's start. All right, thank you. Uh let me share my screen. >> All right, so um yeah, thank you. Um so we'll be presenting today about our app which is the orunit sync app and uh this um this was developed to support orugunit synchronization across different DHIS2 instances. So as you know organizational units so we refer to them as orunits are something that almost every DHIS2 instance um heavily depends on. So they represent a structure uh within which our data is collected and organized. For example, they may be hospitals, they may be clinics, districts, administrative areas or any other service delivery unit. However, as a DHIS2 instance grows and expands and matures, particularly when an organization operates, several DHIS2 instances, for example, at the family health bureau, we have several health related public health related DHIS2 instances and they maintain a the same or similar organizational unit structure across all of these systems. So maintaining these various systems and maintaining these organization unit structures across these various systems become difficult. That was the core problem that we wanted to address. So in the first part of the presentation I will briefly explain why this application became necessary, what problems you are facing and the overall approach we took to solve them. afterwards my colleague will take over and take uh explain how the application itself including the features and the technical aspects were developed. So um when it comes to organizational units, we know the organization units are the real backbone of the DHIS2 system uh DHIS2 platform. But um more more when a system matures or rather the DHIS2 implementation matures, the organizational unit hierarchy becomes quite large and increasingly difficult to um maintain. And even when it comes to health related programs, they may operate several DHIS2 instances within the same organization. For example, like how it happens at the family health bureau of Sri Lanka. So this um the challenges that happen are that there are new facilities that are in um introduced and these need to be added. There are facilities that are edited, metadata changes, parent and hierarchy changes and updates. So all of these changes are manually done. And this manual maintenance takes a lot of time and creates opportunity for inconsistencies to happen. That is where the organization unit sync app was developed. So the few issues that we realized the repeated manual updates. So the problem there was that it reuses metadata from the source instance and because of this it reduces the need to recreate organization units again and to reenter the data separately. So the m the problem with missing facilities and outdated information. We decided to address it by comparing instances and identifying units um identifying units and adding to adding units to update the different the mismatches in corresponding units. This was also sorted out by the organ by using a UID to identify corresponding organization units. Um and then the overly broad changes. So the that problem we decided to address it by giving a feature or um ability for this app to restrict the score and then there can be errors when applying the changes. So for this we introduced a facility where we um implemented a dry run before the execution of the final um um synchronization. we were able to do a dry run so the administrators could see and inspect the inspect and validate the changes before making the final commits and uh finally these were all these results were downloadable in several formats which Mr. my my colleague will explain in detail. So to address all these problems the solution we came up was the DHIS2 or unit sync app. This was a controlled way where we could synchronize organization unit metadata from the source instance to a target instance but most importantly with a review before applying the changes. So we were able to control the synchronization through this. Um in this very briefly my colleague will go into details regarding this but we had a a very simple workflow where we initially established a connection from the target instance to the source instance and then like I mentioned previously we were able to limit the score and decide exactly what organization units to compare and then we finally the administrator was able to run a comparison to review what changes would be applied. This is the validation step. And finally once the synchronization is executed the system would validate then execute the u metadata synchronization or rather the orunit synchronization. So how all of this happens and how this is built the technical overview and the features of our app will be explained by my colleague. So I would like to hand over uh for to continue this presentation. Thank you so much Sach. Uh I will now focus only on the technical side of the unit uh synchronization. how we build the application and how it communicate with the two instances and how this matching logic works and all okay the uh the technical story follows the same path as in the interface in the app. So we connect is using the through the DHS to route app. So I will explain later and go the current target instances through the user existing sessions. Next the operator define the scope and the chosen the identity strategy. Then the app fetch and compares the metadata and presented the results for overview validates and the select operations and then finally execute only confirms its only. Each stage creates an uh lock before move to the next stage that is give the uh kind of safety feature we implemented in the app. Um the core stack is a deliberately small. The interface is as you all know we use the with the react component and white is used for the development and the build workflow. Under the interface, browser side service modules handle the routes and uh uh source request, target request and the access checks, comparisons, uh payload constructions and the synchronizations integrations of course in through the DHS2 web API and uh the DHS2 route API. The source and the target remain normal DHS2 instance. the available implementations we review uh review does not need those component for the core comparison workflow that keeps the deployment simpler and uh removes another place where the credentials or metadata could be stored. So this application hold the working comparison in the browser state and lets the operator export evidence as a CV CSV or JSON file. The important architecture value is uh separation of responsibility. The react application coordinates the workflow while the DHS do remain the authority of author authentications and the access control and metadata persistence. Then uh before opening the uh our orunit or unit synchronization app administrator create the test route using the DHS2 router manager. So documentation available in the DHS2 website. So router manager is the use uh user interface for managing router API metadata. In that we define the upstream source addresses the authentication scheme and any required headers and the authorities and sharing rules that control who may use it. So if you if anyone asks why we use this route app. So you know like organ sync app runs inside the DHIS2 instances. A direct browser call to another DHS to server introduce the cross origin restriction uh and can push source connection detail into the browser application with the router. The browser call the target DHS2 API. For example, the router runs endpoint and the target DHS2 server invokes the configured uh source path. Uh this gives us a single controlled endpoint and the controlled integration point like uh the router can be enabled or disabled then tested and shared through the DHS through administrations. And so the flow is organizer unit uh uh calls the target the target router API calls the source DHS to instance and there is response returned through the same path. The app does not need the operator to type the uh source URL or the credential every time. So it select an approval route. This also means access to the governance decisions. So to to whom can access this uh app and who could not access this app. So when you come to the uh organ uh synchronization app this is the first tab we see in the interface that's called connect. So on the connect tab the application load and enable the routers. If you have multiple routers it will show all the routes in the drop-down uh like this. Uh so you have to choose the correct one. Then the operator say then a successful respond. Then you can click on the test connection and it will check whether this is the valid route or not. Then the app perform the separate checks on the source to read the current user context. This is also one of the important point because uh it will check whether the user has the read and write access. So it will list down the uh using the API call whether this particular user has the privilege to use this app or read and write privileges are available. Uh if you require check trail the application does not unlock the next stage. That's another important part because every stage passes only you can enable the other stage. The normal trust model intentionally a simp. The source is read only for this workflow. So we fetch metadata using our source and confirm the target instances. Uh then the second tab is the scope tab. After successful connection the scope tab load the source organization unit hierarchy. This may the depend on the how many organization unit we have in our source instances slightly delay. If you have a more organization unit, it will take like uh uh time to load entire or unit hierarchy and also it'll load your organization unit levels, what are the groups available and all. This is not only the perform feature and then scope define the intent and boundary of the operation. It reduced the number of API pages also. And uh this optional level and group group filtering can reduce the results further such as selecting relevant clinics group for an example. Yes. Uh and uh the screen also makes the behavior uh rules visible. We can decide to do with the units have no codes. So I will uh whether the same code under the different parent should be the treated as a conflict and whether the parent reassignment is allowed. A hidden uh default would be dangerous here. So these decisions are presented to the operator before the comparisons too. So yeah this is uh okay the most important decision is uh how we identify the same organization unit across the two instances. Normally we know the UID and the codes are strategies we use uh they are the two field that must match. We choose UID when when both instance are preserve the same DHIS2 object identity. uh this is the cleanest case for an example when we metadata was exposed and imported while keeping the orunit UID UID if the UIDs are different uh because of the or units were created independently that's the real scenario the most of the instance are happen like that independently so we cannot simply edit one UID because the UID cannot be edit that field so in that case we should not uh situation we have to choose the code. So in Sri Lanka point of view we has a strong basis for this public health organization unit can have a governant unit code. So for example mo area health facilities and the clinics and BHM areas and schools and other reporting units. The code must be separate governed by our uh self stable and unique across the uh instances. If the source and the target have different UID but the same official code, the app can correctly treat them as a corresponding unit. And uh yeah, so when the operator runs the compare, so this is what happened in the back end. So when the operator runs the compare, the application f fetches the pageated organization unit metadata from the both sides. The source request travel through the route. Uh the target request use the current target API. The app ask for identity hierarchy and the supported metadata field including code names uh descriptions dates or coordinates geometry and all the other informations. It then builds lookups maps using the selected key UID or code. For every source unit, it find the corresponding target unit, compares the selected metadata field and evaluate the parent and the relationships. The output is classified as match. Then we can add, update, move or access in the target or block any conflicts. This order matters. Uh so because it we have to identify must resolve first which one we are going to resolve first otherwise is the renaming clinic might be mclassified as one missing record or on a new record the discrepancy view records which feel differ so the results is expl explainable rather than the blackbox instructions. So the comparison tab so uh intentionally uh review surface this is actually uh the one of the key uh interface in the app. It reports the total source target counts always the number after the score matching units then additions how many we need to updates how many we need to rename and how many or units are access in the target units and uh it will show each category uh can be filtered filtered and search and selected and the operator can open the discrepancies for the individual record. The results can also be downloaded as a CSV file so that you can keep it as a record if this is uh if we if we wanted to perform this one. So we need to download and we can keep it as a record. At this point the applications has not written anything to the system. in particularly access in the target uh is not an automatic delete list and all the target units maybe still have a historical data or valid local purposes. Okay. So the sync stage works only with the selected row because based on our previous uh uh comparison tab we have to select. So when you come to this app, it will come only the selector probe. The first pass is a validation ads are prepared as a DHS to metadata payload and sent with the imported mode to the validate. Then update and movization unit endpoint with the dry run enabled. So and also geometry one of the key area geometry is the handle separate because DHI is to expose dedicated geometry endpoint. So this is handled in a different way. If a validation reports an error or warning the operator sees it before executions. The second pass applies the confirm operation to the target. adds use metadata import with the merge mode. Uh then updates to the parent move to put into the target organization and similar to the operation we do in the maintenance app. So geometry is the posted or removed only when it's defer. So this runner also prevents the update and move for the same organization in one executions. And also the payload construction also uh follow the chosen identity mode as is mentioned here. If it's in the UID mode preserve identifier can support directly parent uh references in code mode the application must resolve the corresponding target uh parent before writing a relationships. Okay. Then finally I will again I will summarize the diagram brings the piece of together. Okay. So on the left uh the browser runs the react interface workflow state comparisons uh services and the export functions. In the middle there's a target in DHS2 instance. The application use the user current DHS2 sessions for like uh target web API calls. Inside that same instance router API is the control gateway to the source. on the right side the uh I put ERMI is the local instance name. So forever the instance which return the uh system information user can take the orinit metadata. I will take you uh thank you. I will take you to a small demo if you type time permits. Uh >> just for a minute show the demo. Okay, I quickly show uh since okay so I'm not explaining how this router manager is going to configure. So here you can see what are the routers available here. So then you will test the connection. You can see the user privileges and all whether this uh uh particular user have the right and read and write access. Then when you come to the scope as I said this a little uh time consuming place because it has to load entire organization unit since we have a bulk organization unit there. So I will I already configured one in our source uh one. So you can filter what are the organization unit or you can simply select use root mean you can use the entire thing or else you have to select what are the organization unit you have to synchronize then I will select for example so one mo and I will only select the clinic so this is filter out and also reduce the API calls between the two instances so the continue to compare then I'll compare Yeah. Okay. You can see there is uh only one discrepancies there. You can see uh 30,370 organization unit in the source and the target it has less number but anyway our match our scope is seven organization unit but uh six are matching and one is not matching. So you can see this and you can see the discrepancies here. So you can see source uh it has there are four discrepancies the name field, short name, comment and the address. So I put test uh but in the target I have put full stop mark in the source and added some comments here but it's not there. So when you merge this and also you can see you can untick if you don't want anything then you go to sync then try and validate then execute the synchronization yes that's allina thank you so much yeah >> thank you so much great if anyone has some questions, you can reach out to the team, of course. Um, and if there's something you really want to ask, you can also write in the chat and maybe the team will answer uh to your question. But thank you so much. Uh, and uh, now we will go to the second app. So, magic glasses 2. Uh, John uh the stage is yours. >> Thanks. Let me turn my sharing on. You can still hear me. Okay. >> Yes. >> Great. Okay. And do you see my screen? Oh, no. Now, >> now we do. Yes. Thank you. >> Great. Okay. I'm just going to move things. Hi everyone. Um, I want to talk to you about app I made called Magic Glasses 2. Um, this is what I think of as beyond raw data trying to turn routine DHIS data into reliable indicators for program evaluation. I will go through why I built MG2, who it's for, some of the technology and architecture, how it works. Hopefully, we'll have time for a live demo, and then possibly I'd like to talk with you about about sort of where we are now with it and and where should the model outputs from this app go. So the thing that one of the things that's different about Magic Glasses 2 is that it's um it's an R application and uh it is now an R package that has a shiny application within it that converts the raw D what I call the raw DHIS2 data into analysis ready epidemiological indicators. It's designed for program staff, meaning malaria program staff, tuberculosis, immunizations, epidemiologists at Ministry of Health, data managers, national disease program staff, implementing partners, um, as well as potentially academic teams that are analyzing routine data and anyone who needs reproducible documented methods. But uh the key is I wanted this to be designed to be usable by by the public health decision makers not just statistitians. So a little background of the motivation for this obviously data is needed for program decisions. You know where are you going to devote resources where to increase supervision add commodities what are the disease trends we need to find out how well an intervention is working. And so people often would go to DHIS2 that that's that's a great source of data, the national HMIS. The problem is that those instances are really shaped by the reporting behavior as much as by the disease and the the the program activities. A couple of of issues sort of jumped out at me when I first started looking at DHIS2 data. One was there's some unclear elements. You a lot of countries have thousands of data elements and indicators and it's really a challenge to find out which are the ones that are are the most useful. There is a lot of incompleteness in that data too. There's facilities that don't report are basically a silently absent from the totals. There's nothing in the totals that tell you how much of the system it represents. So you get the situation where increased reporting can look like increased disease and conversely decreased reporting can be mistaken for disease reduction. Everybody knows there's data entry errors um and unfortunately they can read as signal. You get impossible values that end up at making a total look like there's there's something big going on. And all this can be indistinguishable from the real counts once once the data is aggregated. And sometimes those are flagged like in the WHO uh DQ app that tells the analyst to adjust for completeness changes when there's you know changes when the in when the completeness is greater than or lack of completeness is greater than 5% but it doesn't provide a way to do that adjustment and so that was one of the things that I wanted to try and change. So the question for me wasn't whether or not the data exists but whether or not the data we use is reliable. So I wanted to give you an example of unreliable data. Um this is actual data from a from a country where as an epidemiologist I would look to see you know how well is the malaria program working. So the in this district it shows me the number of first attendances the number that had suspected malaria the number that were tested for malaria the number that had confirmed malaria. And from those numbers, I'd make some percentages like what percentages had suspected malaria, unusually low. The number of the suspected who were tested extremely low given we were trying to provide a test, be sure we had a test for everybody. And the percent of tests that were positive was over 100%. Okay, that's that's not possible. So someone trying to use this data would conclude from it that this malaria program is seriously underperforming or the data is bad or both. And it turns out that really wasn't the case when you look at the data by just adding one more column and that is how many facilities reported each of those data elements. The picture totally changes. See most facilities submit that monthly form. So technically they report it but many leave individual fields blank. So I added here a column that showed how many of the facilities in this county actually filled in for this month this specific data element. The most number of facilities 219 filled in the first attendances but only 158 facilities filled in this number of suspected malaria. only 76 of them put in the number that they tested for malaria, but 143 over twice as many, almost twice as many facilities wrote in the number for how many were confirmed. Well, is it any surprise then that the number the percentage of tests positive looks like it's over 100%. Because twice as many facilities were recording how many were positive than we're recording how many we tested. So a lot of data is there. It's useful but it needs to be adjusted before analyzing. They came to the conclusion that this raw data is not necessarily a reliable indicator and we don't want programs routinely drawing conclusions directly from the raw totals without accounting for how much of the system actually reported. In fact, when I did a literature review, when I look at published papers using DHIS data, only 8% of those reports actually mentioned what the data element completeness was, which means it's really hard to interpret their conclusions. You know, in in DHIS, HMI systems, we see data entry errors, we see health workers strikes, commodity stockouts, form redesigns, all these things are things that need to be included. So that's why I created magic glasses. Here's a it's maybe not the only solution, but it's a it's a good start at it. And the idea is that we start with the system of record, a country's DHIS2 instance. And then using an API, we extract some of that data and then with our statistical software, we do some of the data munching, time series models, visualizations. And to make it as interactive as possible, I put it into a shiny application. So this shiny um user interface is a front end that would help the analysts explore different adjustments uh depending on how you look at the data. And all of these functions are also available outside of that shiny app for someone who wants to do a the bespoke analysis. The the uh MG2R package has a front end with this R shiny. It also includes uh uh dependencies on other R packages like like leaflet for maps, DT for tables, ggplot and plotly for visualizations. There is several analytics engines. There's uh in time series forecasting there's a number that are that are listed here. also uh use a number of web API uh packages. And then lastly to deploy this what someone would do was they would install R on on their computer. They would then install the MG2R package and they could run it locally or on an R server like um connect cloud or something like that. When the app opens, it has a a workflow that's sort of what I think of as a guided pipeline to to would take that raw data and then extract defensible estimates from it. And the way they would do that would be go through sort of step by step a metadata review. That way they can identify what the key elements are and look at their attributes like does this data element allow you to record zeros? Then after you find out what you want to look at, you then download that data and it gets imported into the app. And there's a broad DQA summary section to give you an idea of how complete this data is, how many outliers are in it. And then there's a a a follow-up page to really dig into the reporting and try and identify among those facilities are reporting what we'll call champion facilities. I'll I'll show this to you. those that are that are consistently reporting and whom probably we should rely on for assessing trends. And then we can dig into the outliers. There's more than one way to figure out whether or not there's outliers or at least to to suggest them. Many people use a mean and standard deviation, but there's probably better methods. And in this section, we provide um six actual sequential algorithms for identifying those outliers. And one of the key things then is we then have a a the possibility of of censoring those values. You can choose which outlier detection you want to use and then you can make sure that those values are no longer used in the analysis. Then we have a page specifically for doing time series models um which is a great way to evaluate whether or not an intervention has made a difference. We have u a newer section that allows one to figure out what's the total burden of illness. Meaning what happens if if all facilities were reporting all the time, what would we can we estimate is the actual number of malaria cases that are in the country. We also integrate with climate information. And then as a last um uh final part that's sort of in development, where does this data go? And I'll demonstrate this for you right now. Okay. Oops. I got to switch to this page. And here we have the welcome screen for Magic Glasses. I just want to confirm with everybody you you now see the app. >> Looks good. >> Thanks. So, uh, the welcome screen points out that this is set up with a variety of pages. These are, uh, shiny modules that are are passed into the app. The first shiny module is a setup where you can select what country instance you're looking at. In this case, I'm using the um one of the demo DHIS2 demo instances from PDROWO. Uh I would love to be able to show you some some examples from the countries I work with, but but I don't think that's that's appropriate. So, we'll just use this this general one. This shows me that I have several downloads of metadata and a bunch of dead is already here. The metadata is perhaps one of the most uh useful things coming to a uh looking at at the system. Lists out all of these all of the data elements, information about the categories, the data sets, and an opportunity particularly for someone who's first looking at trying to figure out what's what to actually view the form that's there. Um it's always a risk at doing a live demo here, but let me just see if I hit Yeah. Click on that and now I can see this is the monthly form that's being used filled out. So I have a a much better understanding of of how this data becomes available. And then I won't go through all these but I do want to point out that my list of indicators to scroll here I have information on both the numerator and the denominator which is often difficult for most end users to see. And it's by looking at this that you can really know exactly what what the data is. Uh information on the regions. See that there's an opportunity even to to pick out specific um uh districts and then zoom in on on the data. All the subsequent analysis can be based on on say this particular district. if I were to uh select it. The next step for the user is to define what data they want to include. I'm call them in this case formulas. In this case, there was and you can one could have a a list of them. They're saved as spreadsheets. In this case, I have a malaria formula that lists uh 23 different uh data elements that I think would be useful for analyzing the malaria program. Now when I go over after that's uh downloaded and entered into the the system I then look at the um data quality and let me pick one of these to look at. Let's take all cause outpatients. And here I can see that um all cause outpatient data element was in 2022 it was only filled out 75.9% of the time whereas now it's great progress is filled out 95% of the time across the whole year. So that that's quite good. But it also means that if I'm looking at any trends in this data, it's going to be biased by this increased reporting. So I'll have to do something to control for it. I can look I can um look at a little more detail about which areas are reporting more or less etc. Then I can get a little overview of uh outliers of this data. There are generally very few outliers. This top line here shows that 99.5 to 99.6% of the values had no error flags. And when there were errors, they were generally very small, less than 2% of the total value. There's a number of other things in here that I just in the interest of time I won't go through but just to keep the process going I then zoom in on total uh reporting oop that's in patients let me just oop summary here we go so here is a more indepth view of the reporting on all cause outpatients and this I have listed here both the histograms of the number of reports and then the total values that are reported in this page. It's showing with the black dots in this graph the actual number of reports that were received each month. And as we talked about it goes from roughly about 75% up to 95%. So how am I going to adjust for that in analysis? Well, one way of doing that that we favor is to find those facilities that did report each and every month during this whole time period, the so-called champion facilities. And that's what's shown here in the red dots. And you can see over on this chart in 2022, it's listing how many uh there's a histogram of the number of reports submitted by each facility. Ideally, they would have all submitted 12 reports for the year. And you can see that um there's the red of the champions, but there's a number that didn't report each month. And interestingly, you can see that there was oh almost 600 facilities that did report every month in 2022, but not all those facilities reported every month in 2023. So we end up with this smaller subset that reported each and every month. And that's a that's a a critical cohort to use for the rest of the analysis. Then there's another page here to zoom in on the um let's quickly go back to all cause outpatients and we have a view of oops uh live demo issue. I've selected the wrong thing. I'm just gonna be sure. Why is I meant to have time to go through and get this uh fully set up beforehand. Let's see. Go to the evaluation page. Here we go. So, what we end up with is a page that has restricted to data for facilities that reported consistently and outliers removed. and then the ability to apply time series model to it. So let's say imagine that in January 2025 we had uh done something to try and increase participation at um health facilities try and make sure there was greater utilization and we want to see what happened over the over the 12 months following that. Was there an actual increase in uh healthcare utilization? The this is now doing a series of time series models. It's validating them. It has found a best fit model and now doing an um estimating what the what it expected the uh outpatient attendance to be had there been no intervention or no change. And you can see that with this model there actually is very different very little difference from what actually happened to what might have been expected. So even though you see that sort of increase there we would conclude that there really isn't a great difference and you get sort of some statistical measures here. there's a slight decrease. And using a um a distribution of change model, you see that um since this is all stochastic, there's lots of different permit uh we do f this does 500 different replicates of how the forecast could go. And in about uh 80 85% of those replicates, it shows a slight reduction. probably not even that important. But anyway, this provides a very robust statistical um output for someone trying to do an analysis. And as I mentioned earlier, we have a couple of other pages for uh looking at the burden of illness, climate um and there's even a newer uh option under development to have an AI assistant help you look at the data. Okay. So I the last thing I just wanted to point out uh now you have some idea of what what this app does is the question of well where do we output all that data as it exists now in the shiny app. Each of those charts has an option to be downloaded. Um and so that could exist on someone's laptop. that could be included in a presentation, but we may want to have it put in back into a DHIS-2 uh instance that there's all these different but there is so much output. The question is what would we put back in it? So we're thinking about a couple of output options. If you know DHIS2 is a system of record and MG2 provides some reliable analysis, where do we put it? Well, one option would be to make a companion DHIS2 instance, sort of a data repository that it could link to. There is the chap model option that is where we could have these models put back into the original DHIS2 instance. We feel like there's issue of governance with that though. Would somebody want all of these different models put back into the system? It's one thing for climate which is a very standard um data element but it may be different for uh a program to say yes I want to have all these these malaria or other models. And then there's right now there's the status quo which would be to have the output as separate downloadable um uh files and that is for instance if someone were looking at uh this chart just by pressing this button they would they would get this image downloaded to include in a in a presentation. Okay. So, uh, thank you for letting me give that overview and I'd be happy to, um, answer any questions or clarifications. >> Thank you so much. Um, Gassum, was your question answered or not? Or would you like to >> I believe it is. So, uh, though I think I would also add to, um, is there like a feature for like a custom report design? Uh I see we can download the charts which you answered >> and then the question is because on DHS too like in the apps a lot of people ask actually about like a how can we create like a custom report and stuff. >> Yeah that's an interesting idea. I'll take that up as a suggestion. I always think that yeah at the end of a session after I go through this sort of pathway it might be interesting to have a button that then downloaded an output from each of the separate pages. It went through what you selected um the data quality output. It could make a a standardized report that might be a convenient way of of documenting a whole session. >> Thank you. Thanks. Yeah. >> Do we have any more questions? >> No, doesn't look like that. So, thank you so much, John. Thanks also to his Sri Lanka team for sharing about your applications. These applications you can check out, try out, give uh creators feedback. If you have any question, also reach out to them. As you can see, you could suggest some improvements that maybe they haven't thought about, which is great. So, it's a win-win. And now I'll be handing over to Kai. So, Kai, the stage is yours to kick off the AI forum. >> Great. Thanks all and really cool presentations both both from uh John and His Sri Lanka. Really cool uh applications and excited. Yeah, I don't know. I'm impressed by both of them what they can do. But um yeah, let's go ahead and kick off this new AI forum section and and these developer meetups. The intention is to um have a space to to kind of talk about relevant current subjects and and using different AI tools because it's a little bit of a kind of uh frontier. It helps to to talk about these things with our peers. Um and so the the format for this will be we'll have um Damore from his Pwanda share a bit about uh uh his experiences with setting up a local AI model to kind of process DHS2 data. That'll be the subject for today. Um, I think that this is a a common desire because people have uh particular needs on how their their data is used. It's not always okay to send um like sensitive data to a commercial AI operator over the internet. And so there are some cases where a local model running is desired. Um, and so that's something I think people have been interested in. We'll we'll start by having Demor talk about uh his experiences with that and then we'll open up to a discussion. Everyone is invited to ask questions and and share their own experiences afterwards. Um so yeah, that's what we'll do for the next uh half hour. So over to you, Deore. >> All right. Uh hi everybody. Can you hear me? >> Yeah, sounds good. >> Okay, perfect. Uh, let me try to share my screen. Uh, oh, sorry. Okay, perfect. Uh, so, uh, hi everybody. Uh my name is Zamur and I'm coming from history. I'm with my team Maurice and Mutari. So today we are presenting HMR digest which is a local a local AI or tool calling on DHS2 analytics. So um what we actually we've been up to in the last uh few weeks uh we built a tool calling agent inside the HIS2 and basically this is not like a chat boot or a text to sec it's basically uh a tool calling agent which plans it acts upon what what what tool to call and it observes the results and then it gives you answers with sources and it can Repeat this over and over again until it it it gets the actual answer with the the actual citation like this is the kind of data element I use. This is the indicator UID I use and here's the the results that I've got. So think of it like a junior digent analyst who opens up you know DIS to and you know runs a perfect query and uh at a high level this is how it looks like. So basically uh a user uh opens the app through the HIS2. So it it it has a H UI and then uh depending on the prompt that the user has has asked then the orchestrator has to decide uh uh through using uh a local NM it has to decide which uh tool to call and then uh there are multiple tools that do multiple functions whether it's running analytics whether it's just uh pulling like the number of children with vaccine and Then uh it outputs the results and then the uh we we leverage the the HIS2 user data store to keep the the user chat history and maybe uh as a simple uh flow uh so there's a question through the HM digest app and then there's the orchestrator which decide which tool to call but of course there's a on the governance layer there's the grounding guard which you know uh uh has to understand the prompt and uh adhere to the to the uh governance instructions or maybe uh it rejects any unverifiable number. So if you ask how many children have been vaccinated in this region uh in this specific period I mean like it has to pull out the right number. So it always rejects unverifiable number as uh we'll see back in the demo and then uh you know we get uh an answer with cited um source the source I mean the indicator UID I mean you have to you can actually look through the syncing process and verify if the uh the application actually called the right uh indicator or uh uh that element and also you So uh we have like we store the the user history in in HS2 user data store. So um uh I mean I I uh I saw a number of questions where people were asking why like why running locally and of course as uh K just uh said there's the data so where I mean people don't want to put their uh data or uh the analytics to their cloud hosted frontier models and uh there's also Uh there's also the cost element because uh you know like these are like you can imagine how how much tokens you you can consume just by running one query because you're calling multiple tools. So you can understand the the you can see the amount of work that goes into the number of tokens you can consume. So there's also uh the costing uh element but here it's one time server installment and then you just maintain the your infrastructure. Um so the element of the bandwidth so there's no cloud dependency. You can actually host this uh uh model on your local infrastructure and also uh this is a a DHS to plugin uh application which inhalds the logged in user. So there's no elevated uh permission. It's it's par log based access. So you cannot actually access the data that you're not supposed to access. Uh I mean there's a so in the settings there's a you can I mean you can connect to a frontier model but it's optional and uh we clearly state that your data are going to leave your your network. So it's a it's a sort of a disclaimer. We just put it there as an option as we see in the demo. So uh among the things we've tried as we were building this through the process I mean at the start we on the llama we we used queen uh 2.5 which was like a starting point but then we started other uh we we we tested other models like queen 3 queen.3 which has more um more capability like reliable tool calls and optional thinking and actually uh looking at the infrastructure that we had it was like a perfect model because we we we this application heavy rise on on tool calling. So we also explored sort of a big models but you can understand it goes with the infrastructure and here we were in the development uh mode and uh there's the uh the frontier model that you can plug in as an option. So like one of the lessons we learned through this process this heavily lies on on on tool calling. So whatever model you choose you uh you choose to use it has to be really good at uh uh tool coin functionality. Um so currently this is where uh our model uh this this is like the the infrastructure in which our model is running. So we're using uh eight uh virtual central processing unit with 24 uh GB RAM. I mean the RAM is we I mean which natively lands on the on this port and um uh and then the the DHS instance we can reach the the DHS instance through the path over the TLS. So the query runs with uh your own uh authentication. There's no a separate authentication because it just uh uh pulls the the user uh the the the user authentication. Um these are some of the practical challenges that uh we faced and also uh we we've run a a number of uh uh testing experiment where we we just handed it over people who who work with the implementations and u these are some of the things that we so there's obviously the CPU latency uh I mean you can imagine we're running everything on 20 24 GB RAM So usually the the response is between 90 70 and 100 110 seconds I mean for complicated uh comp complicated queries. So so there's also the the issue of u RAM sealing and no swaps I mean the model has to fit in the memory and I mean that itself it's a limitation for us because uh otherwise it will just crash. So we we have to rely on a a small model but which has uh which is really reliable in terms of tool calling. Uh so um and the recommendation is actually uh it's better to run the model on a separate server uh than the where the DHS2 instance that you're running because I mean there's a separation of concern and you don't want uh them to compete for memory and the CPU and and this is how we we keep it trustworthy. So there's the grounded uh guard rays where actually uh we've managed to tighten such that especially on the on the on the responses like every answer has to be grounded up like you can actually trace back in the thinking process and you say okay did he pull the right uh uh indicator did they did pull the right uh data element so and it has to reject every uh unsettated uh numbers and we can we can also track the exact metrics that uh the the system used. Um and the security I mean the in terms of uh the user access the application it's a it's a DHS2 plugin so it has the the DHS2 itself uh handles the authentication and uh the login. or uh where these are like the the the safety cases that we we've taken care of. Uh and maybe these are some of the uh insight that we we've gained through this process. Um as we're building as we continue to uh to build other versions. So it's interesting to rely on tools rather than just maybe uh you're using a rug, you're just uh uh retrieving uh some information because I mean uh DHS2 is he like if you read the if you read the DHS2 API documentation it's well documented. So you can run on tools and understand like natural understanding of the LM and it makes your system really powerful in terms of what it can actually generate. So also uh we recommend actually encoding like most of the stuff are encoded in the code because you don't want to to hope that LM will always um uh understand uh the raw details of your instance. So most of the things are most of the things are actually uh run through the cord or through the the prompt engineering that we did and uh to go against is the the fabrication of now let's say you ask like I want to know how many you know malaria cases in the last 3 months of course you don't want to you don't want your system to fabricate any number so it has to really beat have at rejecting any number that is not cited or it's not sure of. If it's not show up there, maybe a data element you it should maybe you know pull uh all the data elements that are related to to your query or to your uh prompt and then maybe you can select to be sure that actually it it pulls the right number and uh you have to have you know a good budget for your CPU so that uh you keep the latency high and also the separation of concern where like you have to you're running your your your DHS2 instance but also there's a separate CP uh server for for this system because I mean it it's hosting the alarm and also the model itself and and the server the CPU has to it has to the the model has to fit on the CPU. So you don't want them to compete for uh for uh storage and the CPU. Uh uh so um this is some of the uh some of the some of the things that we we we rely on like of course we're not building we not trying to replicate things that can be actually taken advantage advantage of in dishes to these are some of the things we do in terms of actually uh identifying anomaries or incomplete data I I mean we can lie on the detistic statistics that are already building in the DHIS2 the Zcore to to uh actually uh know the difference between a delayed data or maybe a gap in terms of uh report. So uh I mean this is the ongoing work and it's something that we really want to to uh prompt engineer well in our system such that uh we can actually uh uh solve the the challenge of data quality in DJS2 instance. So um this is it maybe I can uh uh I can jump on the demo and then uh we can open up uh questions. So uh this is a testing this is the testing sense one. So basically I start by uploading the the application. This is so if uh let me refresh you come under. Oh not that. Okay. So again I have to Let's see if the application. Okay. So again I can start by uploading the application. Um, and then again HM. Okay, there you go. So you can open the the application. So basically what the first thing it does tell us has to index your instance because I'm already you know signed in. So you can see it was able to pull like what like what instance am I am I working with when was the last time the analytics was run and you know everything. So I mean in the settings of course you can come and select uh are you working with the local model? How are you choosing a cloud hosted model like with the frontier models and then you come and uh you test your connection host the prices and then you save then you can come and interact with the with the with the system. So um for the demo purpose I've prepared a number maybe let's say uh if I go very first into visualizer that I mean I can see let's say anemia referrals so maybe let's try to see what's the the anemia referrals like across the country And so it's something I can come and uh ask the the applications like what's the name area for situation in the the last 12 months in the country. So it should be able to uh go and and and do the right thing. So this is the thinking process I was talking about. I mean you can close it if if it's not your interest but for verification purpose you may want to actually verify did it pick the right um uh data element did it pick the right period did it pick the right organization unit. Those are the the kind of things you may be interested in knowing before actually uh believing what the model what the the system is telling you. So on the question of the retance you can see it's really slow but it's due to the uh limited infrastructure the CPU we are running on and um yeah so uh as you're waiting maybe uh yeah thank you and uh yeah you can ask uh any question and I and my team uh will be able to respond if it's something that we can actually answer. Thank you. really really cool stuff and really nice to see your experiences in these things. I especially like this kind of like ability to inspect what's going on and have these these um audit points. I I'm curious about a couple other things, but also is there anyone else in the in the chat who has questions? Um, feel free to unmute and ask. Also, Gossson, you can make any start. >> Thank you. Yes, I was waiting if if anyone wants to ask first, but uh I'll go ahead. Uh, so thank you for your presentation and the demo. And uh my question is I saw when you actually clicked on settings uh it felt like your Olama is hosted on a VPS right so it's not right now it's like on a network it's not like on your device right is that the case >> yeah that's the case >> but if I try to do it like locally um it would still work like a local host >> absolutely I mean like you can see I have like actually on my on my computer for the demoing purpose because uh we had issued this thing into the testing team. So we wanted for them to test on a separate server but it can like absolutely can run on your actually your your machine CPU. >> That's great. That's great. Thank you. >> So you can see the graph came out well. I mean you can do some sort of like verification and and compare what you get but I mean like from developer perspective I mean you can come and check these are I mean these are the UID of the determent it's it fi you can see it understand that the last 12 months and you know it needs to break it down in month three and so these are the things that you can go through and say actually It's really understanding what's happening behind the hole. >> I see hands raised up. >> Okay. >> Yeah. Go ahead, I >> thank you. No, this is a this is a really impressive demo. I uh I'm really impressed by the the things that you've done and it seems like it's uh uh one of the more mature implementations we've seen of uh of this and it looks really good. I actually have a couple of questions but I can I can start with one of them and then I I'll let others chime in on other questions. Um, I was just wondering, I saw when you opened the app, it said something about it had uh indexed your instance and I was just wondering what uh could you explain a bit more what that index is and uh where it's stored if it's somewhere else. >> Sure. Um so basically what this what happens uh whenever you upload this application on your instance uh the first thing it does is actually understanding a metadata because it's not like build for a specific instance I mean uh we we we encoded the there's a tool that actually uh sends a request to understand uh the your instance what's their organiz uh organization structure, what are their data elements, what are their uh what are the indicators, what are their data set data in your instance. So every before actually it's because I I ran it before but once you shoot it up on a new instance the first thing it does is indexing and understanding your metadata so it has context of whatever you're going to be asking. That's uh that's very cool. Is that um is that stored anywhere or is it mostly in in memory when you start app? >> Uh you mean there I'm guessing uh I'm guessing the index that you build is some kind of uh like the structure itself on how everything is set up and everything. Is that do you like put that index in index DB somewhere or or is it uh L20 >> so it's not stored anywhere. So think of uh so there's a an API endpoint where you can actually understand uh the metadata of your instance right so that's the endpoint that we we we reing actually and so the first thing it does sends the API request and understand the structure of instance and then uh it stores that in your uh user data store which is like a a very small Yes. Okay, makes sense. No, thank you. This looks great. >> Maybe I can take a turn with the question. >> Uh yeah, the I I think it's smart to have like the model kind of run some decisions and then have tools kind of handle the uh the like API connections. Um what's the kind of architecture? Where do the where do the tools live? Are those in the in this DHIS2 app or are they like a separate service? And um does the model have any actual direct API access to DHIS2? >> Okay. Um so uh the tools are built within the application itself. So um it's like an agentic tool but then uh it's it's you can plug in into um any DJ S2 instance. So these tools are are built within the application and um once the application has read the metadata and the structure of your instance then um uh a user when the user ask a question then it's the role of the LLM to actually first understand the question and then it has to decide uh which tool am I calling depending on their understanding of the prompt. So it's not stored anywhere. Everything is is built within the app but then can be uh plugged in within the HS2. >> Okay. Yeah. But the yeah the model lives outside somewhere else. It's a separate running service, right? >> Yeah. Yeah. The model the model is a Yeah. It's a plug-in. It's it runs it's just for natural language understanding basically. But the rest of the thing is actually uh like tool typed function. >> Yeah. Cool. Nice. Yeah. Cool. I'll hand it over to Gossip. Go ahead. >> Uh, thank you. I think I have two questions. Um, the first is, do you think it would be possible for instance to limit the LLM's access to for instance aggregate programs and like uh not allow it to access tracker uh programs? Um, and actually when you're running this LLM, does it take like per permissions to read exactly what the uh user has like does it require a super user to run this? I'm sorry if I missed it. So that was my question. My first question is about is about permissions and access and uh the second question is this all read only? So there is no right. You can't it cannot change stuff, right? Is that Thank you. >> Yeah. Okay. Uh uh that's a great question. So uh remember the model is on your infrastructure. It's a open-source model on your infrastructure and it knows what knows then. So um what does basically understanding your uh your prompt and then it has to translate that prompt into you know API like the right API request. So for now it has the the on read functionality and um um it interacts with the analytics tables by the way it doesn't it doesn't touch the DS2 uh database just works with the analytics tables and um so the tool it's like it's like I mean what what the person does in in when they open visualizer it's whatever table whatever navigation button you pull it's an it's an API endpoint you hit so the mode is just hoping the application to understand okay so you've asked this question what's the best translation of this uh prompt into an API request so it has no edit or do something kind of functionality within I don't know if that answers your question but uh again uh I'm not alone there's a couple of people in my team so they should feel free to jump in and uh maybe respond that to whatever maybe I may have forgotten. >> Natalie has a sense. Oh, sorry. Go ahead. >> Thank you. I was just saying it does answer the question. Thank you. >> Okay, cool. >> Thank you everyone. >> Yeah, thank you everyone. Uh I want to add that uh uh for in question uh yes they uh for now uh uh this app treats only data so it doesn't even index any uh any track metadata so it doesn't know for now and uh for for it does it does only read uh this uh metadata for aggregate and analytics and uses the response is just pull out the aggregated data or any other metadata which used like for example if you want to know how many health facilities are there it can show you but doesn't uh create a new indicator or create an advertisement of organization so it doesn't change anything on the on the DHS side. Uh uh the other thing is that uh the idea was that uh for example for an instance which has been there for so long you don't know exactly what the metadata or how they call their data element. So but you have a general idea for example there is malaria cases. So you put in whatever you think and then it proposes you among the the metadata exist uh the the it brings you what's there and then you can uh select and then the visualizer it sects the best visual available and then you can navigate for example here you have a chart but you can turn that into a table and maybe D can click on table. It brings here a table or if you want like a a bar chart, it will bring you bar chart. If you want to sort it will do that. Then you can have a quick quick view of the DHS2 data. Uh, of course, if you're not you are in a hurry or you don't know how to use DHS to uh no more no more anal. Thank you. >> Cool. Thanks. >> Addict, did you have another question? Yes, but I don't want to hijack the meeting either. >> You're next in line. >> Okay. Okay. No, I was wondering um do you guys have any uh uh eval setup, any evaluations or anything that sort of tests the agent in different conditions or is that on the road map? Uh yeah so uh it's something that uh we're actually thinking of what's the structure like the best structure where we can actually evade this this system other than relying on the guard rays and uh checking actually uh reverse looking at things that it it was able to pull. So the first thing we've done is actually you know putting it into the hands of the implementation team and they're doing their best as you can but we're thinking of the structure evaluation mechanics that we can apply that is structurally and standardized. >> That sounds good. I think it's it's one of the things that we've sort of been thinking a bit about on how to improve uh improve being able to run evals. Um I'm not sure if we have anything to to share too much about it, but um I think it's quite important for for work like this that you're able to at least in a safe and secure way do it. Yeah. Impressed. >> Absolutely. Thank you. >> One last question from me. Do you think it'll be possible to uh like petition the implementations that you're working with to get the resources to to host like a an independent server to run your model on and uh maybe get a GPU to run it. or is it still in the kind of like trial phase? >> Uh maybe uh can you come again? >> You mentioned um that it would be best if the the model can run on its own server instead of the same one that DHS2 is running on. What do you think it'll take to get the resources to set up a new server for uh the local model? Yeah, I'm sorry. Go ahead. >> Uh uh for the people who have tested so far they are they are positive about it and we have like two groups one and which uh uh will look for a server where we can put put this local local uh but for one specifically because they have a contract with anthropic so they have available token for them they you use anthropic. So uh we have those two groups and uh all of them are are willing to find space for this server. >> Okay. Yeah. Are they do they send their their DHS2 data to anthropic servers for those services? >> We we do understand the kind of partnership they have and it's something that they have in revealed to us, but I mean like we're waiting to see if there's some sort of like special arrangement that they have or we we we clearly don't know. Gotcha. Yeah. Okay. Well, awesome. Uh, yeah, thank you guys for presenting and I see that we're kind of coming up to the the end of the session now. That's really cool to see. Thank you for for sharing your experiences. Excited to see more about where it's going and uh yeah, good design on these things so far. Uh, if anyone else has uh questions, feel free to reach out to the Hist Rhonda team. There's uh let's see I can link the thread that Damores uh opened to present these things in the chat here also um to claim your cop badge you can um use the link that Gossson posted in the chat and uh yeah I think we'll we'll wrap up for today. Uh anything else Lena? >> No. Thank you so much for joining. Thank you presenters and see you next month. Bye-bye. Have a lovely day. Thank you.