August 2026 Developer Meetup: Behind the Scenes: App Competition 2026 Review Process & more

So, hi everyone. Welcome to this month's uh developer community meetup. We have a couple things that we'll uh that we'll share with you today. Uh first, my colleague Yuan will u share about the the review process that we did for the app competition this year where we had a lot of cool community apps submitted and and several that we're really excited to kind of elevate and share and um kind of how we went through that. they'll tell you more about that and also the process that we have when we go through um submissions that we get on the app hub which are a little bit different but um uh share a lot of the s the uh share a lot of similarities but um also yeah a similar process and then after that I'll share a little bit of an announcement of something that we'll be doing in upcoming uh community meetups which will be a uh a regular slot just to have kind of open discussion about AI subjects. It seems to be a a common interest just to talk to other people about these kinds of things and ask each other questions. And so I think it'd be valuable to have the space to talk about this. And I'll uh I'll share a little more about what what that'll look like at the uh at the end of this meeting. And uh so think about the kinds of things that you'd be interested in in hearing from other people about uh what are they using, what are they doing, what what's working, what's uh what pitfalls have they run into. Uh so yeah, I'll talk a little more about that at the end and then for now I'll hand it over to Yuan to talk about the the app review processes. >> Yeah, thank you K. So I just go ahead and share my screen. I hope you can see it. Okay. So yeah, I'm just going to talk a bit about what the review process was for the app competition or the the app competition rather for the conference. um how we went through everything and also uh I would then go into the um app hub submissions that we are getting as well and the review process we have when we get these new apps and how it's changed in the last year or so um I would say with the increased use in like aentic coding uh and how we use agentic coding ourselves but also how we see others use these kinds of tools for the apps that they submit. So, first off, the app um competition. So, thanks to everyone who submitted. Uh the technical quality for the this year's app competition was really high and the top five or even like 10 candidates were really close in like total score that I got and the submissions range from like admin toolkits to AI metadata tools. Um we also had some like Android clients as well and even our package as well. So a lot of interesting and varied stuff. So we got in total of 21 uh apps and extensions submitted. Uh what we aimed to do during the review process was to use around uh 30 minutes per app but we started the entire thing by just going through all of the submitted demo videos together. Um so we sat down went through everything and this was usually enough to give us an indication of uh what kind of apps we wanted to look closer into um and and so on. So we went through this initial list um and then we picked out from that list uh what kind of apps we wanted to look at more closely. So with this process we ended up having between two and six independent reviews per app which we are very happy about. So the the final results is really a good indication of what we actually um think about the apps. Um so we split uh the score metrics into three as well uh where we could score each metric from one to five and I'll get a bit closer into this uh different metrics a bit later. So the review process itself is really quite straightforward. uh we had this huge um Excel spreadsheet where we have listed all of the apps. So the first thing that was done was that the reviewers picked an app from the list of submissions and as I mentioned the initial reviews were done together in a team um just to give us a feeling of what kind of apps we wanted to look into and how we should split the review process between us. Um and then once the app was picked, we would open the submission details. Uh that this is data that was connected to the submission itself. So the GitHub link to the source code um some of the apps were already um listed on the app hub. So this also included an app link uh and a demo video as well. And the 2-minut demo video was usually the fastest way in and enough to give you a feeling about uh especially the design and usability of the app and also the impact it might have. But for the technical quality, we would often have to go into the source code itself and have a closer look. Um we also created this review template that you could use which was basically a list of um checkboxes. So you would then copy this review template into your own notes and just walk through the list uh of the checklist and um you could just check them off as you went along and then you could use this checklist to um score your three metrics basically. So it was scored based on the technical quality of the app, the impact it might have on the DHS2 um yeah the the DHS2 community and also the usability and design of the app and then you simply just loged your review. So the reviewer added a row on the review list with their own name, the app, the three scores and a short comment on anything unusual. So um as I mentioned the three scores, it was impact like what does this app enable for DHS2 users and how would um DHS2 team get real value out of this app? um does it fill a gap the platform leaves open? Um is it generic, is it useful and so on. And then uh the technical quality as well. Um the general code quality but also the maintainability. Um so an example of this is if there are any like hard-coded values or if there are separation between the configuration of the app and the source code itself. Um, so we looked for like um hard-coded values within the app. Um, if uh it's tailored for a specific use case or if it's been made more generic, for example. Uh, we also checked that the app platform is being used and uh DHS to UI library for example. And here we often use aentic tools to help. So we could do a quick scan of the entire repo and give the AI agents a list of things to look for basically. So instead of having to go through everything manually uh we could get like pinpoints to where we should look. So in this way uh agentic tools is really powerful and can save you a lot of time. Um the last score metric was design and usability like the ease of use and visual design. Uh also the quality of the documentation itself and how easy it is to follow the readme to install everything, configure everything and test it on the different versions. Um, and the final score that the apps got was the mean across every review this app received. And yeah, I would just like to take the time to thank all of the reviewers. We actually got more than 60 total reviews, which is great. And also for the domain specific apps, we asked the product teams to add their own reviews which they happily did. Uh so big thanks for to all of the product teams as well to take the time and go through many of the apps and also thanks to everyone from our team who took the time to log the reviews. So we have Lena, Kai, Musafar, Gasim, Claude and and many others as well. So big thank you to to everyone and also congratulations to DHS2 audit vision for winning this year's competition. Um well deserved and yeah there were a lot of good apps this year. Uh so I don't know if Edson is here but yeah um clap for for Edson. really good job. Um, but sadly we also were a victim of a malware attack during this review process. So, um, I just quickly wanted to bring it up just to give a bit of context and to why we have updated uh, some of the review process we're now doing for the AppHub um, submissions. So, we noticed that some of the submitted apps were unfortunately infected with malware. Um, I just want to stress this that this was not the submitter's fault at all. Their machine had been compromised without the knowledge and the app itself carried it along. Um, so given the scope, we were lucky that only uh one machine got infected. Uh sadly this was my uh machine. Um so all that I did was to open an infected app in VS code. So I did not install the app I did not build the app and this was enough to get infected. So the trust boundary now is not installing the app. at the moment you open the folder in VS Code. Um, so we just put uh the review process on hold until we had everything under control and we were able to scan all of the submissions as well. Um, but sadly by the time this was identified, the malware had already accessed all of my Google accounts, my GitHub tokens, and my SSH keys. Um, so I thought that I was able to clean my computer, but sadly it had already downloaded child processes that were running dormant on my computer. Um, but all of this has led us to a more much more strict security process during app reviews. So before we even touch the code, now we do a complete security scan. um preferably just read only directly from GitHub without pulling anything from GitHub and down to your um local computer. Um but yeah, this also resulted in the Pollen rider attack where my GitHub got hijacked and basically just pushed all of the same malware to as many branches it could reach in the shortest amount of time. So, um if this happens to you and if you're able to spot it, I would recommend you to um first off remove yourself from any organization that you're in on GitHub and then um turn off your computer completely and contact the DHS2 security team as quickly as you can. and you would have to uh reformat your computer that is affected. It's not enough to identify the process and kill it because by the time it has already installed child processes that are um not easy to to dig up. So yeah, the the security routines have been um upgraded after this attack as well. when we go through all of the apps. So this also leads us into the appub submissions. So this is similar to what we did for the app competition. So the app competition is kind of a lightweight version of what we go through each time a new app is submitted to the app hub. Um we use the app hub submission guidelines as the template for the review. So this acts as the list of [snorts] like checkboxes that we go through during the process. Um and we are quite strict now with what we approve directly into the app. um because first off it's a signal that it's safe to install and it covers a gap in the DHS2 ecosystem that is not uh currently bridged by other apps and many instances doesn't have the capacity to check an app themselves. So this listing on the app hub acts as a security gate um as well. So if an app is on app, it should be safe and trusted by the HS2 team. That's the core idea. So this review process has changed a bit the last year. Um earlier we just went through the app hub submission guidelines and uh did everything in um like one sitdown and we created this extensive review document that we would share with the developer once we've gone through everything. Um but now since the number of submissions had gone up um probably in some ways due to uh use of agentic tools um we have uh altered the review process a bit. So now we have this four-step approach where we um have like this set of priorities. So the first item is the most important and um if we run into a blocker we reach out to the developer immediately. So we no longer create this um full reviews. We try to have a more open dialogue with the developers uh of the apps early on in the process. Um so the first thing is the security scan which we've incorporated now um after the last attack. So the whole repository is scanned in a sandbox before anything is built um or open in an editor. So again we use agentic help here. There were a lot of good tools that got made during the last attack. Um so we used some of those scanners now as well to help um identify attack vectors, common attack vectors like injected payloads and all of these things. So yeah, injection tooling hooks, um dependency tampering and all of this. So a clean scan means it's safe to build and if anything is found we stop the review process immediately and contact the developer. Um and then we basically follow the readme um as it's stated in the source code. Um so we follow it directly and uh go through the setup instructions on all of the supported versions. of the app. Uh and then we also test all of the documented features and check that everything is working fine. Um, and basically if we have trouble configuring things and if we see that you can't um install the app directly from the app hub without having to manually tamper with the source code. um we give this feedback um as well because it's important that you are able to um install the app from the app hub and it should work at least in some way. Um you might have to do some configuration but this should be handled through the app itself. Uh you don't have to go and and edit the the source code. So uh this is just a test on how well we are able to follow the readme and if we run into any issues ourselves and also if you have like third party services uh integrations that you have to connect to this should be really transparent and what kind of setup instructions there are in place for those kind of third party services should also be declared in the readme. So you should know um all of the different um services the the app is talking to and how it's done. And then we check for the code quality and again we use some uh aentic tools here. Um we check for the use of the DHS2 app platform, the app runtime, uh the DHS2 UI library. Um we also see now more than before that um if people are using agentic tools to help develop the app, we can see some uh structure and leftover code as well. Uh like dead code that is left in the source code from previous um features that have now been surpassed. uh unused configs um also documentations for features that have been removed and reference to documentation that has yet to be released. So um things like that we are looking for and also a really important thing is the security in the code as well like using roots for external services. Um I will talk a bit more about this later. um using no credentials in the browser and also no hard-coded values in the source code. Um and lastly, we go through the the rest of the items in the submission guidelines which uh are not too important and can be easily fixed. So usually for apps we we end up either here or here now before we have to give feedback to the developer which is great. So these are like the simple um checkbox items that are on the app submission guideline. So does it include screenshots? Um how is the app documentation? How is the app name, app icon? Um, is the description honest? Use of external services, transparent impact, genericness, and and so on. Uh, and as I mentioned already, uh, we've had to alter the review process a bit. Um we see that now with people using more agentic tools the code quality is often uh really good. Um so this means that we can be a bit stricter as well on code feedback now since agent can address those fixes in a day where it used to take developers a long time. So one example of this and what we see a lot when people use AI agents is that the agents often define their own components. So we see a lot of custom CSS uh and a lot of custom solutions in the submitted apps. So one of the feedbacks that we usually give is to use a consistent design system and we do um want people to use the DHS2 UI library of course. Um so this is one of these kinds of feedbacks that we tend to give almost for all of the submissions now. Um and we also see like the general number of submissions have gone up quite a lot. Uh so as I mentioned we used to compile this large feedback report but now we are having more of a dialogue with the developers and we hear back from roughly half of the submitted apps. uh which is why we have moved away from this uh full-on report to this kind of dialogue as well because if we don't hear back from them then um yeah we can't really uh approve the app until we've [snorts] had updates. So um we also see a lot of apps now with AI features as well like AI built into the apps um chat bots tools that helps set up the just metadata natural language queries to third party registries um and so on. Uh but if you use this kind of tools and you connect it to third party services uh you have to be really transparent about how this data is handled on the other end. If it's saved, if it's stateless, um how can you actually verify this and who is handling the data on the event? Is it saved somewhere? Um uh and so on. So uh the DHS2 admin must be able to opt in basically if there are some third party uh connections going on. And another interesting thing we've seen now with [snorts] the use of agentic tools is that some apps are submitted that is not connected to DHS2 at all. So it's not connected to your instance. It's not using the API at all. Um, so it's basically a standalone app within the DHS2 global shell and everything that it does is to connect to a third party service. So we have to be a bit careful about the kind of architectures we're seeing and and stuff like that as well. So if you want to create an app for the DHS2 app hub and you want to be uh approved quite quickly uh this is the kind of u sheet you should follow. So, first off, follow the app hub submission guidelines and check that you have ticked all of the boxes. Basically, um test the documented workflows for all of the versions that you are supporting. Um be mindful of the sign system. Uh preferably the DHS2 UI library, but others are okay as long as you are um consistent in the use. Um we see that some uh organizations have their own u um UI systems and we have more or less um approved these kinds of apps since they are consistent in how they use um the different components. Uh we also checked for use of the DHS2 app runtime um and test app and all supported versions and that it's transparency about the third party services and integrations as well. So if you follow all of this um and that uh the source code doesn't include any malware of course then you could be quite sure that you uh have come a long way in getting your approved. Um we um now that roots are available in all supported DS2 versions um apps that connect to external services should use them. So uh browser based credential handling is no longer accepted on the appub and we are quite strict about this as well for new submissions. Um the deadline across the app was midjune 2026. So we will now begin to review the apps on the app hub and find out which apps needs to be migrated to using roots instead of um browser based credential handling. So if you know that your app currently stores or fetches third party credentials in client side code um please prioritize migrating to use roots. Um thank you. Um that was all that I had. Um here is the appub submission guidelines and the roots migration guide if you want to go for from uh browser uh credential handling to the roots. Thank you. Are there any questions? I saw that we had some uh messages in the chat. So we had one message about one of attack and yeah. Okay. >> Sweet. Thank you for the presentation. on interesting to see the pro see these processes and and what's new in the world of the appub and uh also thanks for the the incident report. Uh yeah, it does sound like a an important thing to share as my Miko said with uh with the community and give a heads up because that that does seem to be like a an attack surface that uh I think most people don't think about. U so cool. Thanks a lot and feel free to to write any more questions in the chat if you have them about uh the AppHub process or the app competition or other things that we've seen and then in the meantime I will present uh a little bit. Oops. Was that the right window? Yeah, great. So, um yeah, I just wanted to share for future. Sorry, one second. Let me get my Zoom set up. There we go. Yeah. For future dev meetups, um, something that we want to do is have a space for people to kind of just have free form conversations uh about AI subjects that are on your mind and would love to talk to someone about. This was inspired by um [snorts] a lot of conversations that I had at the annual conference this year that a lot of people just come up to me and ask what are you doing here? How like what what agents do you use? Like what processes are your team using? How do we like connect these with instances? Um, and I feel like this is something that uh is a world where we're all kind of exploring and there aren't many like textbooks written on it yet. And so it's a lot of individual exploration and and trial and error and reading articles and things. Um, and it's it's nice to have an opportunity to get together with the, you know, a group of other developers and and talk about what you're doing because it's a you know, we're in a we're in a similar context together. Um and so yeah to support a space for that the um yeah the plan will be to have discussions in these meetups uh um a little bit free form. will will come with a subject um kind of sharing on a particular or yeah one subject uh as a as a start and ideally we'll invite someone to just kind of kick off something that they've they've used that they think is effective and they can talk about that a little bit for maybe uh five or 10 minutes and then other people can ask questions but it also doesn't it also doesn't have to stay on that subject if someone's got a burning I'm sure so many other people do too and it's useful to talk about that and collect information in this kind of peer-to-peer environment. Um, so this will start next month. I think we'll set aside about 30 minutes in the in the session to to talk and we can kind of explore how that goes. If if the conversation is going uh really actively and we want to have more time, we might uh either extend this or or make a separate session uh from the developer meetups. [clears throat and cough] But um that's where we'll start and see how it goes. uh anyone who kind of does this kind of kickoff thing and and shares something that they're uh uh that they're interested in or has found effective will be um recognized with a community of practice badge. we'll make one for uh AI forum contributors and um in the meantime also between these sessions always remember that you can post in the community of practice um and and make any subjects we we can maybe like keep an eye on those and bring those up each month um you can use the AI tag to make it a little easier to filter but uh probably in the development section if that's where the [clears throat] there are other sections in the the community of practice to kind of filter down uh where these uh where these conversations go and have them easy to find. Um, I have a couple ideas for what these these kind of base subjects might be. And again, they uh we can go kind of wherever we want to. It's it's not strictly Yeah, it's it's not a rigorous format. The idea is just to to get together and have a venue to talk. But um uh but also think about some things that you're interested in and I'll collect them in a second. We have a little interactive form that I'll put up and you can share ideas um with uh with me and the group here for what you're interested in. But just as some examples to kind of get you started thinking for the for the next section here with the with the form. um things like effective prompts that that you're using and and what gets you better output with agents when you're when you're doing code things. And uh this is something that I've done actually. I've I've looked back into my prompt history and seen what what's worked and what's needed like updates or follow-ups to kind of fix um things like useful skills and plugins that you found to do what you want. the the for example the grill me skill has been uh seen as pretty useful in different contexts here in the DHS2 core to help spec out uh a system for kind of setting an agent loose on it. Um plus we have some other skills that we've made for um DHS2 apps for example and are looking to make more. uh anything that you found for effective feedback loops? I know of of uh things that an agent can run and then test something uh practically to see if it worked and then kind of correct itself and do those things because there are ways that you can set that up. for example, with test instances to like run a real app and uh with real data and stuff, things like PR review processes, things that you found effective to um for your team to or for yourself to get good feedback consistently that what finds what like gen genuinely adds value and what can be a little bit of a waste of time. Um, and also something I've been hearing a little bit about, uh, one of my colleagues has set up a cool system to, uh, and this is this is, uh, Ulav, my colleague, um, isn't here in the meeting today, but he set up a cool system for, uh, spinning up, uh, instances to test against, and we'll spin up like instances of several different versions to make sure an app works against all of them. um and things like that and kind of like workflow tools and um maybe things that you've set up working with production instances or uh but also things like um uh ways you've used models inside of apps effectively. Uh what uh what kind of things work well on on mobile in your experience? Have you set up a model running in your organization's environment so that you don't have to send uh like a data out to um a a commercial entity out on the internet? Um there's lots to talk about. Um and it it'd be great to share experiences. Uh, so we can switch over to this um this little interactive poll here. I'm curious to see uh what kind of subjects you would be interested in hearing about. You can use this either go to um menty.com um and type in that code or you can scan in this uh QR code to open this up. There's two two prompts here. One is um what subjects would you be interested in hearing about and and asking your uh peers and colleagues about? And uh after that there there's a question um about is there anything interesting that you've experienced that you'd like to share and uh what those things are. Feel free to write as many things as you want. Nice. Really? Yeah. Really interesting subjects here. And yeah, a lot of these um definitely uh yeah, big questions like risks of blind use of AI and sustainability of AI developed solutions. That's something that's been coming up a lot in conversations. local models and open uh open source AI tools, people's developer rules. Yeah, that's uh looking to the future is definitely um a big subject and hard to predict. professional AI development versus vibe coding. Yeah, that's something that uh we think about a fair amount um in these things and uh I think we're we're in a time where the development tools get better and better and so like vibe coding will approach kind of professional stuff. Um it's interesting to to wonder where things uh will go and where it'll reach. Um in the summer we made the assertion that it still takes professional developers to make um production worthy things. Uh but it it could conceivably happen that with the smart enough tools and uh and models >> with and guided well enough >> could become >> like successful. >> Someone have a question there step-by-step demo of things useful. Yeah, I agree. like um we only ever get to see our our own workflows and there could be lots of different things that we uh that are effective out there. Um so seeing some of those things would be useful. All right. Nice. And then uh in the I'll just leave it on this slide since I think um let's see there's more mitigating malware using AI. Yeah. Um there yeah I think there's two sides of that. There is um AI models running uh to do development things can potentially install packages that it doesn't it doesn't vet and uh supply chain attacks have have grown a lot in the last year or two. npm security is is stepping up more um I think as a part of that >> and so when when agents are installing a bunch of packages into a a project those can be potentially like risky without uh like a human kind of uh vetting them so that that's one area that an AI could potentially trip up. Um but also there there's the side of like using agents to do like useful security scans. there are kind of security oriented models that are um that seem to be getting pretty useful and effective and uh have been used internally in DHS2 for some for some uh um useful research and uh yeah so there's um yeah those are those options so this is great thank you for these suggestions I'll uh yeah, feel free to post more. You can you can email me at kai dhhs2.org if you want to message me directly or you can message developers at dhs2.org. Um if you have more ideas or post on the community of practice and um I think these are a nice start. We'll we'll pick some from these to kind of kick off the next subject. Um, and yeah, thank you for your input. I think that that um we'll wrap it up for today. I think we can end a little early. Thanks everyone for uh for sharing your ideas. I'm excited to kind of uh start this up. Gossam has a helpful post. Uh remember to pick up your developer uh developer meetup attendee badge in the community of practice. And thanks Yuan for the very cool presentation. Nice to see those things. Thanks everyone for being here and uh we'll see you next month. Have a good rest of your day. >> Thanks everyone. Byebye. Thanks everyone. M