Submind YouTube summaries
Thumbnail for Fix AI Agent Latency: CPU, GPU, and Edge Placement | Jon Alexander, Akamai

Fix AI Agent Latency: CPU, GPU, and Edge Placement | Jon Alexander, Akamai

Watch on YouTube

Video summary

In the realm of traditional cloud computing, performance discussions typically revolve around CPUs, storage, networking, and memory, but the conversation shifts significantly when addressing AI, where GPU capacity and token generation speed become the primary focus. While metrics like time to first token and throughput are indeed critical for model efficiency, a deeper analysis reveals that up to 90% of an AI agent's total task execution time often occurs outside the GPU. The majority of this latency is actually generated during the tool-calling phase, where agents interact with third-party systems, invoke external APIs, and retrieve real-time data or context necessary to formulate answers. Consequently, optimizing solely for GPU performance is insufficient because even the most powerful models running on the latest hardware will fail to meet user expectations if the surrounding infrastructure lacks proper connectivity to these essential tools and data sources. To address this challenge, Akamai emphasizes the importance of an "AI grid" that extends from centralized data centers all the way to the network edge, allowing for distributed token generation in real-world scenarios. This approach requires sophisticated orchestration to manage workloads across various physical locations effectively. Large models that demand clusters of powerful GPUs are best suited for centralized locations to ensure high utilization rates and manageable maintenance, whereas the CPU-based components responsible for running agent sandboxes and executing code should be placed closer to the end user. By separating these distinct computational needs, organizations can optimize end-to-end performance while simultaneously controlling costs, ensuring that the specific workload is located in the most appropriate environment for its requirements. The core of this strategy lies in the ability to coordinate between CPU and GPU resources across a distributed infrastructure to minimize overall latency. An AI grid orchestrator plays a pivotal role in this ecosystem by scheduling execution effectively across different components of the application, ensuring that the CPU and GPU operate in harmony rather than as isolated bottlenecks. This separation allows developers to place the heavy lifting of model inference in centralized clusters while keeping the interactive, tool-calling aspects of the agent near the user's location. Ultimately, achieving low latency for AI agents depends not just on raw processing power, but on strategically placing different parts of the application architecture to drive down end-to-end response times and deliver a seamless experience that meets modern user expectations.
Read the full video transcript
In traditional cloud, we talk about CPUs, storage, networking, memory, but when it comes to AI, we mostly talk about GPU capacity and token generation speed. What metrics are missing from that conversation? The ones that actually matter for real-world AI agent performance. >> Yeah, no, the the the the this is kind of a key optimization that we've been looking at and again as as we've been working with customers is that they're they're building the these agents. Um the GPU is really important like the end we've been deploying a a lot of the the Blackwell GPUs and the speed of those GPUs so uh time to first first token, the throughput, um the number of tokens they can generate per second, those are incredibly important metrics that that we're seeing that customers are are optimizing for. But for these applications in the agentic application, often what we're seeing is up to 90% of the overall task execution isn't on the GPU so isn't kind of generating tokens. The majority of that time is actually in the tool calling portion of the agent. So calling out to these third-party systems, calling external APIs, pulling data into pulling in context, pulling in real-time uh information that's needed to generate the answers. This is what's actually generating the majority of the end-to-end latency for for the task to be successful. And so optimizing for the end-to-end system is what's important. So GPUs are a really important part, but often what we're seeing is there's a a really big portion of the end-to-end time that's not on the GPU at all. So you can have the most powerful fastest model running on the on the latest GPU, but if your compute is in the wrong location, um if you don't have good connectivity out um to the tools, if you if you're not running um with a good connectivity to uh storage and other data systems that you need your system is still going to be slow and isn't going to meet the expectations of of the users. >> Akamai has been talking about a compute continuum for a while now. What does that actually look like in practice expanding from centralized data centers all the way to the network edge when we talk about AI? >> This is a big optimization problem. So something that we've been working on is what we call AI grid orchestrator. So Nvidia is talking a lot about their kind of AI grid reference architecture. So as we move from kind of centralized token factories to kind of these real world systems that can deploy applications for kind of real time token generation. Having the ability to have that in a grid to distributed infrastructure is important. But that needs orchestration. So you you've got all of these different um locations where you can run workloads. And what we find is there are certainly very large models that need to be deployed on powerful GPUs often clusters of GPUs. Makes sense to have that in more centralized locations. So you don't want to have those deployed in hundreds of locations. It's very hard to maintain utilization of highly distributed clusters of GPUs like that. But then what we see is for the um for the CPU where the agent is running where we're deploying sandboxes where we're spinning up containers to kind of run kind of code execution. That's something that is isolated in sandbox to an individual user and individual session and it makes sense as we look to optimize end-to-end um performance manage cost for that to be close to the end user. That's where what we're seeing is the separation between CPU [clears throat] and GPU is often um um in these types of uh distributed systems, we can place those uh different components of the application in the right um physical location to drive uh end-to-end um latency down. And then also make sure we're driving effective utilization of that infrastructure. Uh and then key key requirement for the orchestrator is to coordinate between the CPU and the GPU and schedule the execution effectively across those different components of the uh of the uh infrastructure.