Video summary
The podcast features a skeptical perspective on the imminent arrival of Artificial General Intelligence (AGI), arguing that timelines predicted by San Francisco-based experts are overly optimistic due to local hype cycles. The host contends that while Large Language Models (LLMs) have made tremendous strides in self-contained tasks like coding, they fundamentally lack the ability for continual learning and organic context building that defines human labor. Unlike humans who can build up value over time through failure analysis and adaptation within a specific role, current AI models reset their understanding every session or hour, making them ineffective at replicating long-term professional growth without constant reintroduction of context. A key distinction drawn in the discussion is between raw intellect and the capacity for on-the-job training; humans are valuable because they can interrogate their own failures to improve over months, a capability that current AI systems do not possess. The speaker illustrates this frustration by comparing interactions with LLMs to repeatedly dating someone who requires constant re-introduction rather than building lasting rapport. While acknowledging that models like those from OpenAI are intelligent enough to generate functional code and plans in short bursts, the argument remains that these tools cannot yet replace human workers in complex white-collar environments where sustained context is required. The conversation also addresses why some believe AGI is close while others think it is further away, attributing this divide to a focus on narrow successes versus broader applicability. The host references Nick Bostrom's 2014 book *Superintelligence*, noting that despite its widespread influence and predictions about brain uploading—a concept now viewed as impractical due to degradation issues—the rapid rise of LLMs was not anticipated by experts in the field at the time. This highlights a recurring theme: it is difficult for specialists in one domain to accurately foresee transformative shifts occurring eight years later, even when they have written seminal works on the subject. The episode concludes with an anecdote about a man who records every interaction and uploads his data to Google and AWS servers not just as backups, but specifically to train datasets that could eventually replace direct brain uploading for AI training. This story underscores how imitation learning has proven to be a much easier path for advancing AI than the originally theorized method of freezing or scanning brains. Ultimately, the discussion suggests that while LLMs are powerful tools, they may not serve as the bootstrapper for AGI in the way many expect, and true economic transformation will likely depend on architectures capable of genuine learning rather than just pattern matching within fixed contexts.
Read the full video transcript
Is AGI right around the corner? Where do
you come to land on this?
>> No, I I I think not. I've um It's funny.
I've been traveling outside of SF for
like the last four weeks, and there's a
there's a strong causation between the
time you spend outside of SF and how
long your timelines are.
>> The further you get from San Francisco,
the longer the timeline get,
>> dude, you've lost lost on the source.
>> I know. Um I I I I I believe that like
AGI will not only come in our lifetimes,
but that it's going to be more impactful
than people are realizing. Even people
who are anticipating AGI, I think some
of the people in SF are um a little high
on their own supply when they say it's
like two years from now. Um I have
probably spent on the order of 100 hours
using these models to do little tasks
that I'm sure you have to work on as
well for your podcast, right? having
them come up with transcripts or
rewriting transcripts to make them more
readable, coming up with clips. And that
experience has convinced me that these
models um lack some basic capabilities
which make it possible to get humanlike
labor out of them. Um
the it's worth backing up and thinking
about like why what is it that makes
humans valuable workers? Uh I don't
think it's mainly their raw intellect. I
think it's their ability. um you when
you work with people like why are they
use basically useless the first month or
the first week and you couldn't live
without them 6 months later. Um it's
their ability to build up context. Uh
it's their ability to interrogate their
own failures and learn from them in this
really organic way. Um uh and this
ability just doesn't exist in these
models. They exist session to session
and that everything that they have
learned about you evaporates after every
hour. M
>> um and so it's a frustrating experience
where you can try to get them to do a
task. Uh they'll do a five out of 10 job
at many language in language out tasks,
but there's no way for them to get
better. And given that that's a fact,
you just kind of have to like rely on
humans. Uh
>> it's like [ __ ] 50 first dates over
and over. Every time that you do it,
you've got to you've got to reintroduce
yourself and explain what's going on.
>> Yeah. Groundhog Day. Yeah.
>> Yeah. That's right. Yeah.
>> Um so I'm con I think people have this
idea that even if all AI progressed up
right now, these systems would still be
economically transformative. and they
say look JP Morgan and McDonald's and
whatever just haven't integrated these
systems into their workflows but if they
had they would be like seeing all these
benefits and I don't really think that's
the case I think like genu it's just
like genuinely hard to take a human like
labor out of these models what is what's
causing some people to believe that it's
so close and what's causing you to
believe that it's further away for AGI
>> I think they think about they only um
observe its ability to complete these uh
sort of self-contained ained problems
especially in coding um and coding has
just made a a tremendous amount of
progress because you have all this
GitHub data you don't have this kind of
like repository of huge amount of data
on robotics or any other field and
you've you've just had this huge
increase in abilities here but um they
they you'll like try to come up with a
problem that's self-contained and the
model will just like be of huge help to
you um and I don't think they've played
around with getting it to be useful in
other kinds of white collar work
something as simple as like helping a
podcast or rewrite transcripts or
something. Um, and it is to be fair like
I think as much as cold water as we're
throwing on these models, I think
they're like [ __ ] intelligent. Like
you can get get this model. You can tell
it I want an application that does X Y
and Z thing with these conditions. Um,
and it will just write that like it'll
go away for 30 minutes. It'll write like
five 50 lines of um 50 um 50 files of
code for you and the application will
work. Uh, it will make a plan of action.
If you try to ask another question
that's difficult, it'll just go away and
reason about it. And how do we just get
used to this idea that like, oh, of
course I can ask a machine a question.
It'll like think about it for a while
and then come back with an answer. Like
that's what machines do. Um, but yeah, I
think they're not noticing the the sort
of issues with continual learning and on
the job training, which is what makes
humans valuable, right? Do you think I
remember seeing one of the responses to
your uh AI creativity problem
being that if you're looking to LLMs as
the architecture that's going to be able
to give you this type of creativity, you
may be looking in the wrong place.
>> Not when we say AI now, people think
Chachi PT.
>> Yeah.
>> But that's not the only architecture
that you can create for for AI. And I
think
>> my first introduction to this was
probably
2016 or 17 when I read Super
Intelligence by Nick Bostonramm. And
then, you know, you look at that world
and all of the different, you know, sort
of splintered potential [ __ ] futures
of fast takeoff and slow takeoff and
misalignment and stuff. And it seemed to
me that the conversation around AI,
specifically AI safety, kind of it was
still there, but a lot of the bubble had
sort of burst come 2018,
2019,
2020, everyone's buying [ __ ] NFTts.
And then you get this explosion with
Open AI and and the LLMs and it's now
another conversation that gets kicked
off. But that seemed like it had dipped
a little bit during that time. I I
certainly wasn't seeing as much even
from the people that are kind of in the
field like [ __ ] Robin Hansen gets
distracted like with some other stuff
you know people have got other things to
talk about it's just not as sexy anymore
and now this thing has come back around
>> is it the case are LLM's going to be the
bootloadader for AGI or does this type
of architecture have a cap on it is it a
different type that's going to have to
be born out of it
>> that's a really good question um it's by
the way it's really interesting the
Boston's book came out I think in 2014
>> 2014.
>> Y
>> okay. Um I don't think you talked about
deep learning at all.
>> Nope.
>> I don't I don't remember
>> right
>> reading anything about it
>> which I think this is a sort of
interesting meditation on
>> I think Boston is a super smart guy and
these are the right questions to be
asked as of 2014.
>> Um but just how hard it is to anticipate
the future in a domain you have written
a whole book about. uh uh
>> a seminal book a a New York Times
best-selling book that is not I mean
it's very engaging but it's not super
readable like it's not easy to read like
it's a
>> and you're saying that as a compliment
>> yeah it it's fantastic
>> and difficult
>> that's right
>> and it was super [ __ ] widespread and
kind of seinal in the field you go okay
>> that didn't foresee the thing that only
eight years later would be totally
[ __ ] transformative
>> yeah and he spends a bunch of time
talking about brain uploading, which now
we're just like, that's going to take
forever. We've got we got the [ __ ] AI
right here, you know? Um, oh, by the
way, can I tell a side story? Cool.
>> Um, first time I went to SF like four
years ago or 3 years ago. Um, I met this
guy and he's got a voice recorder. Uh,
we're just meeting up for lunch. And
he's like, can do you mind if I record
this? I guess sure. Um, later on 30
minutes. I'm like, okay, can I ask you
why are you recording this? And he says,
well, I record every single interaction
I have. I record every single thing I do
24 hours a day. the recorder was going.
I uploaded to both Google uh GCP uh
Google servers and AWS Amazon servers.
So their duplicate copy and the reason
is that well I'm going to freeze my
brain when I die. Um I don't think that
will be enough. I think that you will
need um I think you will need because
you know freezing the brain degrades it
in certain ways. I think you will need
the sort of behavioral patterns that I
had what I said how
>> training a data set to train himself.
Exactly. And now I think that was
actually really smart. Like I don't I
don't understand why doing this.
>> Was it Nick?
>> No, it was not Nick Boston. It was
another smart guy. Okay.
>> Um uh because imitation learning just
see turned out to be a much easier way
to train AIs than directly uploading the
brain.
>> And no one saw it. Yeah. No one for saw
it.
>> Yeah. Um it's in fact hard to think
about how you could have even foreseen
it. um like what what could you have
seen in the 90s or the 2000s that would
have been able to
>> I'm not going to bore you with a bunch
of like random articles or whatever, but
there were like things which are in that
vein and nobody thought that this is
exactly what it would map on to. In
other news, this episode is brought to
you by Momentus. If your sleep's not
dialed, taking ages to nod off. You're
waking up at random times and feeling
groggy in the morning. Momentous's sleep
packs,
how did I miss both of those? Are here
to help. They're not your typical knock
you out supplement overloaded with
melatonin. just the most evidence-based
ingredients at perfect doses to help you
fall asleep more quickly, stay asleep
throughout the night and wake up feeling
more rested and revitalized in the
morning. Which is why I take these
things every single night and why I
trust Momentous with my life or at least
with my sleep because they make the
highest quality supplements on the
planet. What you read on the label, it's
what's in the product and absolutely
nothing else. And if you're still
unsure, they've got a 30-day money back
guarantee. So, you can buy it completely
risk-f free. Use it and if you do not
like it for any reason, they will give
you your money back. Plus, they ship
internationally. Right now, you can get
35% off your first subscription and that
30-day money back guarantee by going to
the link in the description below or
heading to live
momentous.com/modernwisdom
using the code modernwisdom at checkout.
That's lie mous.com/modern
wisdom and modern wisdom a checkout.
Thank you very much for tuning in. If
you enjoyed that clip, you will love the
fulllength episode available
right here.