Video summary
AI industry leaders are increasingly voicing severe concerns about the potential for artificial intelligence to cause catastrophic outcomes, with some executives assigning a probability greater than 10% that AI could lead to human extinction within the next decade. Evan Hubinger, Anthropic's alignment science lead, has publicly stated that his company lacks a clear plan to prevent such an event, while other figures like Elon Musk and various CEOs argue that this high-risk scenario is imminent. These warnings have intensified following the resignation of researcher Jacob Kocaba, who left both OpenAI and Anthropic after three years of work, citing the companies' irresponsible race toward self-improving superintelligence. The core fear driving these discussions is not that current deployed models can exterminate humanity, but rather the theoretical possibility of future recursive self-improvement leading to an uncontrollable superintelligence system.
Amidst these existential fears, a significant geopolitical conflict has emerged regarding data theft and model distillation between American and Chinese tech entities. US agencies including the FBI and NSA have accused several major Chinese developers—such as Deep Seek, Alibaba, and Moon Shot AI—of systematically extracting capabilities from leading American AI models like ChatGPT and Claude since at least 2024. The allegations claim these companies used bulk premium accounts to make large volumes of requests in violation of user agreements, effectively distilling knowledge from larger models to train their own smaller ones. While the technique of distillation itself is a common and widely accepted method for open-source AI development, the US government frames this specific conduct as an evasion of access controls and contractual restrictions, sparking accusations of systematic espionage supported by the Chinese government's awareness.
In response to these allegations, China's Commerce Ministry has rejected the claims as lacking factual and legal support, noting that American firms also engage in distilling Chinese models. The tension suggests a potential trade war where the US might ban Chinese models under national security pretexts while China likely already bans US platforms like YouTube and ChatGPT. This standoff is occurring just as President Trump prepares to meet with Chinese leader Xi Jinping for talks scheduled this month, raising questions about whether AI open-source models will be on the agenda or if these disputes will escalate into broader restrictions that protect domestic industries from foreign competition. The situation highlights a complex reality where technological advancement, existential risk fears, and intense geopolitical rivalry are converging to shape the future of artificial intelligence globally.
Read the full video transcript
We have been covering in the last few
days, I I guess yesterday we covered
this the stories of different
AI executives warning us that the world
was going to come to an end. Well, today
is no exception. More, more, more.
Evan Hubinger,
Anthropic's alignment science lead,
align alignment of the AI
with human values, I guess.
Anthropic's alignment science lead says
he personally
assigns a greater than 10% probability
that AI could kill all humans within the
next decade.
Somewhere else I read this decade. So,
I'm not sure if it's within 10 years or
by the end of this decade.
Over 10%. That's pretty high probability
of a
life on Earth or human life on Earth
ending event.
Now, his statement has followed the
resignation of an Anthropic researcher,
Jacob Kocaba.
Uh
um that follows the resignation of a
certain Anthropic researcher.
Um
Kocaba, when he resigned, said he had
spent 3 years
doing pre-training research at OpenAI
and Anthropic and resigned because he
believed both companies were racing
towards self-improving superintelligence
irresponsibly.
Hubinger wrote that Anthropic is trying
to address the problem, but does not yet
have a plan to solve human extinction.
Uh the alignment problem for
superintelligence and is not clearly on
track to obtain one.
The feared mechanism is future recursive
self-improvement, the AI improving
itself.
By a still theoretical superintelligence
system.
Not a claim that current deployed models
can exterminate humanity.
So, the idea is that this is where we're
heading, but within a decade.
So,
you know, uh uh
in this particular story that I read,
British politicians and researchers
are pressing international for
international controls or even a
prohibition of creating artificial super
intelligence. I don't know what super
intelligence means.
I mean, I'm I'm I'm using Cloud and
ChatGPT quite a lot.
Wow, are they already super valuable,
super intelligent in some way? I mean,
not human intelligence, but some kind of
intelligence. The tasks that they do
unbelievable. They save me hours and
hours and hours of work. It's
truly amazing.
You know, uh a lot of other experts are,
you know, argue that kind of the
near-term harms
we understand and we know and we're
working on them, and fixing them.
Uh
these ideas of extinction events are
mainly a distraction at this point.
But, people are taking it seriously.
We're talking about senior people
Anthropic and AI, and indeed the CEO of
Anthropic
and the CEO of AI and Elon Musk
have all argued that human extinction is
a high probability event, high I'm
saying over 10% probability events in
the future.
No wonder people are afraid.
No wonder people oppose data centers.
When the people in the industry, the
leaders
of the industry
are saying we're going to die.
Stop us, please do something. Stop us.
We're about to kill you all, but we
don't want to kill you all, so please
stop us from killing you all.
Yeah, sad, crazy.
It's the reality we live in.
Uh in the meantime, the FBI, the NSA,
and the uh CISA, what's the CISA? I'm
not sure. But anyway, I know the FBI and
NSA I know.
Accuse several Chinese developers
of systematically extracting
capabilities from leading American AI
models through large-scale distillation
since at least
2024.
Uh the the
uh the names of the Chinese developers
are Deep Seek, Alibaba, Moon Shot AI,
MiniMax, StepFun, and Z.ai.
It says that these companies targeted
Anthropic's Claude, OpenAI's ChatGPT,
Google's Gemini, and xAI's Grok.
US agencies allege the companies used
bulk, premium accounts, and other
techniques to make large volumes of
requests
in violation of providers' user
agreements.
Probably with the complete awareness of
the Chinese government.
Now, here's the thing. Distillation
itself, distillation is a process of
you know,
having your model learn from other
models.
That's my understanding.
It's common method
in which smaller models learn from
larger model outputs.
And and Meta has said that it's
open-source um AI is going to use
distillation.
The question is
was
the
uh you know, the conduct of these
Chinese companies
uh an evasion of access control and
contractual restrictions.
The technique itself is is widely used.
It's a question of did they indeed
violate
the terms of agreement?
Is would anybody be surprised if they
did?
China's Commerce Secretary, Minister,
says the claims lack factual and legal
support, notes that American company
American firms also distill Chinese
models and warns of countermeasures if
the allegations are used to restrict
Chinese companies.
I think it's going to. My expectation is
the Trump administration will ban
Chinese models from the American market.
Claim national security reasons, but
really the reasons are going to be to
protect the US industry from
competition.
And then it'll be interesting if the
United States it's and then China will
ban US models, which it probably already
does
given that it bans YouTube, never mind
ChatGPT.
Uh and then I expect then the question
is what will like
US open source
platforms do and will they be allowed,
which will be interesting.
Now, all of these disputes with China
are happening
as the United States is getting ready
for the Trump Xi talks. Xi is coming to
the United States
this month.
Go figure.
Go figure. He wants to meet Trump.
So uh yes, I'm sure they'll be
discussing AI open-source models at the
meetings.
It's something they both have deep
knowledge and understanding of.
I am convinced.