Video summary
The video features Ari Weil, Vice President of Product Marketing at Akamai, discussing the critical economic realities behind artificial intelligence applications, specifically shifting focus from training costs to inference expenses. While there is significant excitement and massive investment currently centered on AI model training—driven by chip manufacturers, cloud providers, and companies preparing for IPOs—weil argues that this perspective overlooks a more immediate financial reality: inference. He describes inference as the inevitable operational cost that arrives every time an application actually uses its intelligence to generate responses or perform tasks, contrasting it with the one-time capital expenditure associated with building large models.
Weil emphasizes that organizations must fundamentally change how they budget for AI by moving away from viewing costs merely as line items for GPUs or software applications. Instead, he advocates treating AI deployment like a distributed system, drawing parallels to established cloud-native and edge-native architectures that have evolved over recent years. The core argument is that the physical architecture of where inference runs plays a decisive role in determining the final bill; simply procuring hardware without considering the systemic design will not optimize costs or performance.
A key takeaway from his discussion is the need for businesses to become more conversant with token pricing and how system architecture directly influences these per-token costs. Rather than just asking "how much does it cost," companies must understand that their infrastructure choices dictate efficiency and expense long after a model has been trained. By adopting a mindset similar to building complex distributed systems, organizations can better manage the ongoing operational demands of AI workloads, ensuring their budgets are truly built for inference rather than just initial development.
Read the full video transcript
And today we have with us Ari Weil, VP
of product marketing at Akamai, to break
down the real economics of AI inference
and why where you run it changes
everything.
>> Absolutely. So, first of all, it's great
to be back on the program with you
again, and I'm I'm glad to have a chance
to talk about the cost side of the
equation when we think about AI
applications and workloads. And I think
to start off with, I would suggest that
training is really the headline that
everybody loves. It's driving these
massive data center deals, big
investments, and some of the circular
investment that we've seen coming out of
anything from chip manufacturers and
card manufacturers to the major
hyperscale clouds that are also building
out their own infrastructure and
procuring it from others. And even if we
talk about the the landscape of
companies that are looking to do their
first IPO, a lot of the focus has been
on how large of a data center they're
building out to build the next
multi-billion, multi-trillion weight
model.
I think that training uh and and that
headline sort of factor is real, but if
we think about it, inference is the bill
that you get after you get all excited
about training. And it arrives every
single time your model takes a breath.
Every time that you're not training,
either pre-training or post-training or
fine-tuning that model, inference is
what happens when somebody needs your
intelligence to be deployed. And so, I
think the the highlight from our
perspective, where we would start a
conversation, is to say, you need to
stop budgeting your AI applications as
line items to do with just the GPUs that
you're procuring or just the
applications that you're building. You
need to budget like building a
distributed system, which is something
that many large organizations have
familiarity with or at least there are
represent representative architectures
for in the marketplace, because you're
going to hear people quote token price.
How much does it cost per token for me
to generate a workload. But, the
architecture is what ultimately is going
to decide your bill, and that's the part
that people need to get more conversant
on and take some of their cues from
building cloud-native systems and to
some extent edge-native systems for the
last several years.