Architecting Integrity: Responsible Software Evolution in the AI Era - DevConf.CZ 2026

Hi, so my name is Clemon Va. I'm a senior engineering manager at Redat and I'm today uh with PRI um I'm Priy Thomas. I'm director of engineering at Red Hat. So today we're going to talk about um how to architect um responsibly agendic AI system. So this is all um what we thought we think is the best. So um open to um discussions too. Um recently I attended an agentic AI intensive course from Harvard data science initiative. So one of the key takeaways for me was that its focus on mindware. So mindware refers to the rules, procedures, strategies, habits of inquiry and bodies of knowledge that the mind can call on to reason, solve problems and make rational decisions. And the call to action was on updating and upgrading your mindware as you learn. And today with AI, this shift is already happening. The question isn't whether to adopt AI. most of us already have. So the question is how do we adapt our practices to keep up? So we will provide some steps and framework in our session. But the most important message we want to convey is to be ready and willing to update and upgrade your mindware. >> Thanks. >> All right. So speaking of uh of adoption like we'll start with maybe a few questions. So Wu here is already using AI agents. Okay. So when you actually designed or architected those those AI agents, have you just integrated the agents in your current processes like the processes that you had before AI? and raise your hand if on the other way you actually created new processes, new workflow for integrating the AI agent. Oh, okay. So quite good. And raise your hand if you are actually you are an AI agent. Okay, thanks. Great. Um, all right. So as we see yeah like we are all going into this this transformation we are all starting to adopt AI uh in our day-to-day workflow. Um but as we see like um many of us like are just approaching the the problem or the integration of AI agents as simply trying to speed up current workflow or current processes. And um as we take the the human workflow and inject AI into every steps, we expect like a return on productivity. Like often we have like this 10x leap like yeah, we're going to be 10 times faster and so on. Um, sometimes it can feel a little bit like if you try to put like a jet engine into like a horse carrier and it works, but is it sustainable or will it last forever? It's a it's a different question. And if we looked at some data like from a different like report or studies uh we see that yes we we feel that speed and like so in 55% of the the case like we we can actually see like oh yeah like know I think we've all experienced this like when we feel like we have superpowers it's like oh my god like I could do that in so much faster that uh that I would have been able to do without AI. Um but from that speed there is also something that starts to build. It's a lack of trust like so yes we get the speed but do we actually trust the output? So it's it's a little bit like this this balance and and the the faster we go the less trust we have and if we are not careful about this balance we are actually building like technical depth like because we we stop understanding the output of uh of the AI uh generated um work. So it is very easy for um organizations to fall into one of the two anti-atterns which is um one is the AI assisted treating AI like um autocomplete heavy oing human oversight which is which can be slow and then the second trap would be the lights out option right letting AI to run free um fast but unprovable and untrusted. So the third path which that uh that we recommend is the AI SDLC which is H with human in the loop achieves which which achieves both safety and velocity. So the goal isn't to choose between safety and velocity. It is to achieve both um safety and velocity through governance. So this is where we are being intentional about the AI vision and driving the outcome that we want. All right. Um, yes. And as we we start to think about like yeah, how we we we build that AI vision that we want. Um, this comes back to the concept of responsible AI. So what is responsible AI in in my opinion? It's where we know who is accountable at the end like so you can benefit from what AI delivers the speed the scales the consistency and so on but we still want to have a human that provides that accountability at the end and that is responsible um and this is more uh like the concept of having the human in the loop it's something that is more like a psychologically something that we need psychologically to build that trust than like a technical requirement because now technically we could just really have like the agents be autonomous but we we still want to feel like we we still want that control and that accountability when something goes wrong. Okay. So we talked about uh the AI vision and together with pretty we like offering you like a tool to uh help you navigate and create your own agentic AI vision. Um so this tool has like five steps but today we want really want to focus on the step number two and three because we believe this is where uh you get a lot of the value and this is where you can uh get started and spend a lot of of your time. Uh yeah. So why oh sorry good just just quickly like why having that AI vision? It's also like the good moment for you to to pause and think like okay am I using AI just because you know everyone does it or will I will I actually get some actual value out of it. So the the first step that although the second step in in our five-step um um creating your vision so that's um we talk about how do you formulate your value in formulating your value. So you would want to envision the agendic future that and how you would reshape your business. So what is the business outcome that you want to drive? Making this your driving force to formulate your agentic AI vision. And you would want to start by mapping agents to to your into your value chain. >> Okay. And the third step was about like defining boundaries. And I think this is where we start to look back at getting the trust, getting the human in the loop and starting to think about the responsible AI. So for me the step is really critical and this is where we talk about guard rails like how do we check like observability and so on. So as a simple rule of thumb like you the autonomy and freedom that you give to your agents should be matching the um potential impact in case of mistakes. So here we have a simple like four level of agentic autonomy. So starting from the top with AI autonomy which is really like minimal human intervention and if a mistake happens like the stakes are are very low. um guided automation when um AI can be autonomous act act on its own but we still have like visibility and we can still oversee like the process like the observability of of it and then we start to go a bit where as human we keep more control into the human augmentation when we consider the AI agents maybe more as assistants where we drive most of the of the work but we benefit from input from agents. Uh so we still make the decision and so on but we are we are guided and assisted by uh agents. And finally the active control which is where we keep most of the authority and decision making and this is really for processes or workflow where a mistake would be like very impactful. So I mentioned the course that I attended recently from Harvard data science initiative. So during that training I was able to work through an agentic AI uh workflow with this framework that they introduced called agent. So I was um I was sharing this with Clement and we thought this is a practical way to demonstrate the agentic AI workflow and as we talk through each of these stages in the framework we will use a CI/CD pipeline as an example. The disclaimer, don't use this this these steps exactly as we say. We're just managers here. >> All right. So, uh let's dig in and start with the the first phase which is audit. And this is really where you start to identify which process you want to um benefit. Do you want to add like some uh agents? And the key point here is to like start from what you have but at the same time so be honest with what you have. You want to describe your current process not as you would like it to be, but you need to to be really honest about like the current quirks or like the the current pain points that you have, shortcomings and so on because this is where you might find like opportunities to to benefit from from AI. um in our example so coming back to to the CI/CD pipeline uh example. So in the current system we are like automated uh eight uh steps in our pipeline know things that are relatively common from like uh looking at code coverage running test and so on. Um but we have like this uh these steps in the middle the steps number six which is when a failure happen we actually need like a human to step in and to start to look at the logs to understand like is it a real bug in the code that we change or was it a flake with the test maybe with the infrastructure and so on. So in the in the workflow like the manual uh the manual intervention is really the bottleneck uh in our pipeline. So the the the G for the gauge phases requires us to transition from step-by-step process optimization to high level businessdriven outcome thinking. So in the audit stage we have identified the the log triage analysis as a step that we that can benefit from AI. So now in the ga gauge stage we want to understand what outcomes that we are after. So to help us with with this we can use the so what um question or which is similar to the five wise um five method. So we might start by asking we will complete the log reviews three times faster. So what? So engineers spend less time in manual triage. So that so what then? Um zero engineering hours wasted on flakies tests. So this is our outcome. So we want we have identified something we can easily measure. So this stage is really critical since AI is really good and fast at producing output. So we need to be very clear about the outcome that we want to drive. >> Okay. So E so for engineer and like in this stage it's where we we start to go a bit deeper and we start to go into the design architecture implementation and um we also like start to think uh how we are going to break down like the our sequence like original like automation into maybe something more agentic. So for the the CI/CD pipeline uh we we choose in our example to go with five different stages. Uh so the first stage is that uh orchestrator agent that would coordinate other agents like the analyst or the guardians and um the analyst obviously will go through through the log try to classify failures like we can imagine that it would keep historical record of of different failures and um the guardian agent is there also to to actually enforce the the gabics for for each agents. Uh in the stage two, we have like the tasker agents or runner agents and his job is really to run uh to execute and to run the test. Uh we can imagine that we want to run test in parallel in a simulaneous way and when the test fails uh the analyst agent would instant instantly um steps in look at the log look at the maybe historical um memory of all the previous failures and so on and starts to do the analysis uh to uh to recognize like some potential flakes take some potential bug patterns. Um and like in that design I think like what would be interesting is like if the failure is actually known and identify with high confidence as a flag as an infrastructure issue uh the orchestrator agent would automatically like uh ask the tasker agent to rerun the test. So um we could do um immediate rerun and in a perfect world um the actual human developer would not even know that the test had failed uh originally. >> So I think again we can see that engineering engineer phase is where you probably have spending the more the most time in designing your system. Um so for the navigate is where we want to think about how much trust and autonomy that we want to give the agents. So that is where you map the risk versus your efficiency you want to achieve by providing the guardrails and policies. Um so in your in your in your in our example we have two operational modes. So the autonomous agent um where we have the test execution the log aggregation and reruns are entirely uh completely automated because the risk profile is low and the efficiency gains is massive there. So and then we have the collaborative collaborative human in the loop agent where what happens when the agent ident identifies an issue. So we decided to put a guardrail which is the agent is not permitted to autonomously push core fixes straight to the production. Instead the agent produces a summary and a proposed patch and then human engineer steps in as as the final authority to review validate and approve. So this is this ensures a security and trust while providing a transparent feedback loop. >> All right. And finally so to uh to complete like the the framework and the the all cycle we need like that feedback loop uh where we're going to track. So if you remember the outcome we wanted uh this is where this outcome becomes like a concrete set of like metrics that we can uh we can look at. Um and this is also where we can actually prove that by introducing agents into our system we get some value and it's just not just like let's put some some agents uh because it's the IP. Um so in our example uh again to come back to this uh we choose three metrics the release speed uh and like the the goal is to to have like to be able to release faster and faster and to go from maybe something where we were able to release every 24 hours to a target where we can just release as as uh as needed as as much as we want. Um another metrics we decided to to track is like a friction reduction. Uh this is often uh known as toil also where pretty much we want to make sure that um we have as uh little as possible um actual manual intervention on our pipeline. Uh so we would go from maybe a baseline that is like now every time there's a failure an engineer has to step in to some target here we choose like to to try to have like four or fewer manual intervention uh per day. Uh and finally the safety and precision. So we also want like coming back to this concept of trust uh we also want to try to evaluate uh how well our agents are doing. So we want to be able to go back and analyze like the decision that the agents have made and try to see if they actually reported a bug that it was a legitimate bug or if they reported an issue as an infrastructure flake. It was actually an um a flake. All right. So this gives you the the overall framework like it's easy to remember also agents like it's not something like completely new but that gives you actual steps that you can easily follow and uh when you start to think about okay I want to um add AI agents into a certain process instead of directly putting AI into an existing process you can start with those steps and uh like try try to rethink your your your current processes. All right. So uh finally with with pretty we wanted to to give you like a couple of quotes to to end the presentation. Um and our goal is to help you to reflect a little bit on the transformation that we are all going through. And uh as we started the talk with like um we really want to encourage you to be willing to to update and upgrade like that mindware and mindset around introducing AI into into our workflow. And finally, so questions. We on purpose we left quite a bit of time for questions because um this is a topic that moves incredibly incredibly fast and we would love your questions but we are also really interesting to to hear about your your thoughts or experience. So uh we have some times and uh we'd love to have a open discussions if you if you want to share your ideas too. >> Yeah. If not we'll well we are the ones who are standing between you all and the kitty pool so we'll get to there faster too. >> That's true. >> All right. Thank you. >> Any questions? Hello. introducing too many changes at once is usually considered to be uh very difficult and risky. So this looks like completely different uh approaches. Do you have also some suggestions how to introduce by small steps uh in an agile way um where should I start? What should be the first things to actually consider before I get to this uh brave future? I'd probably say start with the audit like I mean what what are your workflows like I mean what is big or small identify that end to end audit like I mean audit your workflows to be able to what is the smallest possible one that you can identify this at least that's that'll be what >> yeah I think like uh one thing we we showed also like it's uh AI is incred incredibly good at creating output like you can create so much output like so fast but so I think our role now it's not so much to think about the output but the outcome that we want to achieve. So like and your outcome can be big or small. So I think I would I would try to to look at like what is the gain or the value that you want to to achieve and like start to maybe break this down into smaller uh smaller bits and pieces and yeah build increment incrementally from there. >> Sorry thank you for the talk. Um I would like to a bit flip the question how we can onboard people to the new system in 5 years where every person is was already uh agentic where the inference uh is pretty small because they can't understand what what's actually going on on the system. So they can't be human in the loop. >> Okay. So just to make sure I understood that. So uh we currently have a agentic system. How do we onboard new people to that system in five years because they won't have the knowledge to actually be in the loop to verify the that the agent is uh is working. That's a good question. I'll leave you think about it. Okay, that's good. I have a small opinion on on this question. I think if this is the future then we did things wrong. If we build our systems or would build our systems in a way that 5 years from now no human would understand it anymore. Then we as humans aren't in charge of the design anymore. Then we would have designed systems or delegated all the decisions to agents. And I think then it wouldn't follow the the agent framework anymore. So I I hope this is at least how my team uses uses agents at at work is that we just we delegate certain work to it but we remain in charge of the decisions. So the output is pretty nice. Uh the documentation is great. people can easily onboard but it requires a certain discipline. So making sure that the agents or when we use agents to write new code a new feature that you know documentation is written about all the choices about how it's being set up how it can be used and those things can be part of the review too. So the following the framework uh merge request shouldn't go in if there is no documentation uh for it. So at least in my experience the way we used agents the open source projects from the documentation side and understanding side got better. The challenge is just that I feel and I saw others on the team struggle with is keeping up with the pace because so much good or so many good features get merged in a shorter period of time than before. So that's more the struggle that I see. I think that would be a good topic for a talk next year. like if you want to think about it and present something, I I I would love to to get some thoughts on this. When I was looking at your example of, you know, creating this pipeline uh with multiple agents, I was I was struck by the fact that when I look at this, I really only see one place for an AI agent and all the rest of these things should be automated, but they should not be agents. they should be, you know, things that are deterministic, not relying on uh LLMs and so forth to do it. Just I mean like the orchestrator is really just a program and the agent's making the decision of whether to reinvoke the orchestrator or not and the guardian should be rules that have been built in with just code. Uh so maybe in the sequence of steps when you analyze your problem you need to make sure that you're not applying you know everything is a nail because what I have here is a hammer. >> Yeah. Yeah. Yeah. That's a that's a good point and that's why we said don't use our example because yeah for the purpose of the presentation we wanted to to try to highlight different type of agents that you could use and so on. But definitely I think part of the exercise and uh actually the engineering step is to to be clear about like what you want to keep deterministic and what you want to to where you can benefit from from the LLMs and agents. That is where the efficiency comes into, right? Like I mean I think what is your like I mean that's the cost of using tokens again versus using just plain clips. >> Just keep the money. >> Yes. And and there's a cost beyond tokens too, right? Because agents introduce non-determinism into your system. So really, you want to reduce the areas where you have agents. If I have something that will work uh just as well as my thought on using an agent, I should use that other thing instead because the more non-determinism I introduce, the more chances something could go wrong in some corner case where this agent misanalyzes something and then that feeds into another agent that magnifies the problem. All right, great then. Thanks for for joining and listening and enjoy the party.