Designing for AI Agency - How AI Agents are Redefining Customer Experience Design

Hey everyone, my name is Matias. I'm going to be talking about designing for AI agency. So I have two goals for this roughly 25 minutes. So first let's understand what aentic AI is if we don't already and then we're going to be talking about three early principles for designing for agentic AI. So who am I? Um one slide about myself. So I've been doing design for the last 16 years roughly. I used to be a design director at a company called IDN. We sold it back in 2017. Um since then I've had my own company. Um roughly I help companies with two things right now. So I help them design what's next. So designing AI enabled products and strategies. And then I also help design teams ramp up in AI. So I just did a workshop yesterday. Who was at my workshop yesterday? A lot of people. Thanks for coming. Um so I've trained roughly 970 designers now including yesterday on using AI at design. So let's start with the story and let's start with this guy. So Elmon Stroger um he had a problem. So his cause mysteriously vanished. So he was an undertaker in Boston in 1890 and his business was l literally dying at that point. So what you have to understand about 1890 and if you wanted to make a call so all calls were connected manually at switchboard. So when you made the call there was an actually person answering the phone asking you okay where do you want to call? Okay. And then they made the connection manually. Um and in the early days there were actually um teenage boys who have often worked at these switchboards. So they would get drunk. They would get into fights at the switchboards. Uh it was a boring job. So it it might take you like 15 minutes to make your call. Um so Oman has an issue. So his calls were actually being hijacked by his rivals. And he had a revenge and it was actually a pretty constructive revenge. So he came up with the first automatic exchange that would automatically connect your call. And over time this manual patching of of calls was automated. and it took several decades to roll out, but now obviously in 25 it's sort of inconceivable that someone would do it manually. How many of you feel like you have way too many browser tabs open at any time or way too many apps that you're juggling? So, I think we're sort of becoming the switchboard operators of our digital lives. We're sort of manually copying stuff across our applications, doing a lot of manual crunch work that's keeping us from our strategic work and actually applying creativity to our work. What would it mean to automate a ton of that work? What what would we have that same invention for our digital work? And I think this is fundamentally the promise of agent AI. So what is it? So over the last three years or so, the world has been really excited about sort of generative AI, which is mostly about answering questions, answering prompts. So I want to do some fun stuff in Lisbon. So here are five things you can do in Lisbon. But the promise of Aentic AI, and it's not here uh it's not here really fully, but the promise is that it doesn't only show you restaurants. So here are some restaurants in Lisbon, but it can actually book you a table at your restaurant that you want to go into. So it's still an awkward early stage. So why is this happening now? So I won't go too deep into the tech side of it, but just to give you an idea what are the sort of main drivers behind it right now. So the first one is reasoning. So large language models at their core when they came out, they were really about next token prediction. so about predicting the next word. But what we're seeing now is more powerful reasoning models that actually can think backward and and forward and in more sort of powerful ways closer to how we think as humans. The other one is context. So these models don't just have sort of a compressed version of the internet. They also have real-time access to user data, organizational data, and web data that they ground their truth in. And they have access to tools. So they can actually um connect to your Slack. They can read if you give them permission they can read your Slack messages. They can send messages. They can connect to your calendar and and take action. So reasoning, context and tools. So the first wave of generative AI has been us as users prompting chatbots and sort of this one encapsulated conversation. But what we're entering now in the second wave with aentic AI is that we as users, we talk with AI agents who help deliver outcomes for us. They can connect to tools, they can connect to context, and they can have conversations with other agents and other u users as well to deliver these outcomes for us. So you're starting to see them in the wild. So um Ser's AI agent is one prominent one. They can actually handle complex multi-step requests from customers. For instance, if you have an order, you're wondering where your order is. It can actually go into the company's systems and track your order and then actually take action on that order as well. So, there's a ton of hype around it, a ton of buzz around it. So, Gardner flag it as the number one strategic trend for this year. Companies were asked about it. So 82% of enterprises plan for agents in the next three years. It was one of the biggest topics at Davos earlier this year as well. So no matter sort of the shape and form of your company that you're working at, odds are in the next few years you're going to get more and more sort of agentic projects on your desk. So if this is true and and agents are coming, how should we think about them as designers? Is it the same as as designing UX flows that we've always done or is there something different? Um, so it's super early and obviously I don't have the definite answers for it, but I want to share sort of three slightly speculative early principles for designing for H&Z AI. So they are useful, collaborative and controllable. So let's start with a slightly obvious one. So value first, harm never. And but when we see a new technology like agentic AI, there's a lot of excitement around it, but it's often driven by this sort of shiny new tool syndrome or or sort of going for tech gimmicks. But I think the the lens and the value that we can bring as UX designers is to really think about what is the actual value that we're delivering. And I think for particularly for agentic AI, I think it's good to think about what is the sort of tedious manual problem that we're solving for. What is sort of cumbersome right now? Um I I I shared the story about connecting man manually the calls. What is the equivalent in your organization or for your customers? So a few examples. So, Zenesk's AI service agent, at least what they claim is it can solve about 60% of customer issues end to end mean meaning that it can actually take action on on the um issues that customers have. Move works AI agent can handle a lot of tedious HR and IT requests end to end. So, if you want um you want a new device, it can actually fulfill that order end to end. And ZIP AI agents can help with tedious work in procurement. So there's a ton of tedious work with uh with contracts with uh with ordering um new uh licenses for instance and sit AI agent can help with all of that. So one way to get started is to test with um no code AI agent framework. So if you have a workflow uh John went deeper into this but if if you have a workflow uh for instance in customer uh research that you're doing you can use uh one of these agent frameworks like Lindy AI to build a simple AI agent to get started. So here's one example. So every Monday go to a backend get you new new user feedback from a sheet let's say it's a Google sheet and then summarize the key insights with AI and deliver it with via Slack to a product lead or to yourself um every every Monday. So you get this sort of continuous customer voice uh insights driven by AI. So the flip side of that is do no harm. So find what is the tedious problem that you're solving. what is the most useful thing that you can solve? But the flip side of that is as these AI systems get more autonomous, their potential for doing harm increases as well. So these errors, they scale with autonomy. So it's really important to be mindful of what are sort of the guard rails that we're building around these systems. So one example is from a company called Hypocratic AI that they do medical agents that offer support. Let's say they can offer you support uh before your surgery, after your surgery, but if there are any critical or diagnostic issues, they always root them to human registered nurses. So that's one example of a really important guard rail. Okay, so useful experiences that solve an actual tedious problem and then thinking about the guardrails around that. So collaborate, what does it mean? And I think this is a more maybe profound and maybe subtle point as well. So I think as UX designers we're really used to going into Figma and designing a deterministic UX flow. So here are the 20 screens that user always goes through and that's sort of their experience. There might be some interaction in it but that's pretty much it. And I think we need to in the next coming years we need to get more into the mindset that we're designing actually teammates. We're designing beings, artificial beings that have their own agency, that take action on their own. What does it mean for us? That's a really profound question. Um, I have a few sort of principles, early ones like I said. So, the first one is sort of this executive assistant mindset that we're designing agents that can anticipate issues, they can clarify things if needed, and then they can act decisively when appropriate. So proactive communication is is one big theme around it. So how do we design for constructive and useful proactive communication? So Alice is a B2B sales agent that can go in and actually look at sales leads and research them and then proactively reach out to them. So how do we design that communication in a way that is constructive and feels natural and and guard rail against against the harms? Proactive recommendations is is one piece around it as well. So for instance, Tendesk's AI agent can proactively recommend products to people uh if they're returning them. So one really important piece is also around the transparency. So as these AI systems get more autonomous, they're going to do stuff on their own. So it's probably a good idea to let people know what they're up to, right? Um so what is the right level of transparency? That's a really important question to think about. So here's one example from Hepia. They automate a lot of finance and legal work with agents. So when you ask them when you ask the system, is this company a good investment? It'll give you sort of a concise answer that okay, based on my analysis, here's my overall recommendation. But then you can double click on it and then you can go deeper. You can zoom in on the reasoning behind it. you can go deeper in that sort of one layer of of transparency and go deeper and see what what was the reasoning behind this analysis and thinking about what's the right level of transparency is really important. We don't want to overwhelm the user but we still want to give them a sense of what's going on. Thinking about different modalities and and how people sort of interact across these modalities with agents is another sort of newish thing that we need to think about. AI agents are getting increasingly capable across all of these different modalities. So what makes sense for your use case? So again, hypocratic AI, they're solely focused on voice-based medical agents. So the interaction is is based on phone calls, and that can be really handy for for people who are not that digitally literate. For instance, another thing that we really haven't had to think about before are handovers. So if these AI agents are um working and they at some point they need to hand it over to a human agent, what is the experience like? What does it feel like? How do we communicate that? Um I was working with a startup called Lastbot um in in Q1 this year and one of the things that we really thought about is is how do you communicate that handover? um they're they're doing a customer service agent and what does it feel like when the issue is handed over to a uh from an AI agent to a human agent or vice versa. How do we communicate that to the customer, the human customer and then how do we communicate that internally to the the human agent? We've never had to really think about those issues before. Okay, so thinking about how can we actually solve tedious real human problems bu build guard rails against against harm and then thinking about how do we design collaborative teammates who are transparent and and do seamless handovers and so on. The third piece is around building controllable things. And the key idea here is that we still as humans, as users stay ultimately in control of these AI agents. So one design decision is what are the critical actions that people humans still need to approve. So here's one example from GitHub's co-pilot and it has a ton of autonomy. So it can go into your codebase, it can review the codebase, but uh there's really critical sort of changes to the codebase that people still need to approve. So what is that right level of approval for customers using your agents or internal users? That's a really important decision to think about. Another one is how do we build control panels? So settings control panels become increasingly important. How do we build ways that are uh to control these things that are sort of the right level of granularity that it's not overwhelming for people but they still have a good sense of of how they can actually control these things as they're going to be doing more and more stuff on their own. One trap or one sort of uh threat that we have to mitigate against is how do we avoid the overdelegation trap. So, at least back home in Finland, when there's a a train that is autonomous for for bits, there's still a lever that the conductor has to push every once in a while to make sure that they're not uh they're not sleeping while they're conducting the train. So, how do we avoid this overdelegation trap? How do we build in moments of mandatory engagement so that people don't fall asleep on the switch, especially if they're handling some high-risk thing? And then how do we build in controls for reliability, privacy, consent, bias, and fairness. For instance, with reliability, it can be going in and seeing what is the data that is behind this analysis. And transparency is getting increasingly important. So I work with a company called SC.AI that enables companies to communicate transparently about their AI use. So you can get really good reports about how are they actually using um AI in their systems. Okay. So let's revisit our three early principles. So usefulness actually finding the tedious problems. What are really the difficult things that people struggle with the manual processes that they have to do over and over again and how can we solve those problems and provide real value not just some gimmicky new tech stuff. And then how do we think about guardrailing against harm? How do we build guard rails that ensure that uh harm doesn't happen on a huge scale as these systems get more and more autonomous? The second principle is collaborative. Really challenge ourselves to think about what are we designing? We're not designing deterministics UX flows anymore, but we're actually designing teammates uh that have autonomy that are active and they actually do things. So having that executive assistant mindset to them, having that right level of proactive communication, that right level of transparency in how they uh communicate about what they're doing and then how do we communicate or how does it communicate about its handovers when it's handed over to another uh human agent for instance. So and the last one is controllable and this is really key as well. So, how do we ensure that people stay still ultimately in control of these things? They're getting more and more powerful every single day. Just yesterday, Anthropic released Claw 4. That's the top of the benchmarks again and it's increasingly agentic and that's going to happen more and more over the next few years. I think our role as designers especially with controllability is is extremely important. So, how do we build in the right approvals? What are the critical tasks that people still need to improve? What is the control panel look like that people set up? And how do we avoid the overdelegation trap and building controls and audits for reliability, fairness, security, useful, collaborative, controllable? So, like I said, it's super early. These are slightly speculative but based on the patterns that I've seen working with companies um discussing with with dozens of design leaders but the velocity is rising and I think one risk or one pattern I've seen with companies is that AI agents they're inherently so technical they're so datadriven that I found that often design teams are sort of left out of the loop completely and I think it's really important us as design community to lead into it to take this embrace it as our new canvas and think about okay we're not just designing static screens or deterministic flows but we're actually orchestrating these agents we're designing things uh and that are hopefully useful collaborative and controllable. So it took automatic operators uh from that invention that we saw it took decades uh for for automatic operators to really scale out. Um the switch to agents is not going to take that long. I foresee it being a 5 to 10 year shift. It's not going to be overnight. I think there's a lot of overhype in terms it's going to change everything overnight. Uh there's a ton of inertia in large comp companies as we know but it's starting now. So if you're interested in agents, you want to sort of understand them more and start thinking about um user problems in an sort of agentic way, this is one handy way to get started. So think about a workflow. So I think the easiest one is a personal one. So whatever personal or or workrelated workflow you have, then think about how could an agent solve this. What are sort of the tedious manual things that are happening in the flow that an agent could do by piecing together a few tools, some AI reasoning, how could this be better? And then use a free to try platform like Lindy AI to sketch it um to build a early prototype of an agent that would solve for it and then then test it with yourself, get feedback on it, test it with with users. So I think it's really important that we lean into this as designers so that we can bring our sort of human and ethical lens to designing AI agents and not just leave it for the tech folks. If you're interested in learning more about AI agents and and how they're shaping uh user experience um and designing AI in general, I have a bi-weekly newsletter. sort of try to keep a pretty hype free and and more rigorous uh analysis of what's happening and what I'm seeing. Hopefully that was useful. I think now it's time for coffee. So, thanks a lot for listening.