Everyone Is Wrong About China and AI Safety — Sebastian Mallaby Explains

Duration: 7:02 1115 transcript words EN summary

Video summary

Sebastian Mallaby's recent trip to China revealed a surprising level of engagement regarding AI safety among Chinese technologists and leaders from major companies like Huawei and Ant Group, contradicting Western expectations based on past diplomatic failures under the Biden administration. While American observers often assumed that China prioritized rapid technological growth over ethical concerns or feared technology only in political contexts similar to the Cultural Revolution, Mallaby found that Chinese stakeholders actively discussed preventing catastrophic risks such as cyber attacks, bio-weapons, and chemical weapons. This shared concern stems from a common desire to prevent rogue actors, including criminals and terrorists, from acquiring dangerous AI tools, suggesting that despite geopolitical tensions, there is a fundamental mutual interest in regulating the proliferation of high-risk technologies rather than allowing them to fall into uncontrolled hands. Mallaby draws a compelling parallel between current AI dynamics and the Cold War nuclear era, arguing that while superpower deterrence might prevent direct conflict through balance, it does not effectively stop non-state actors or rogue states from causing harm in a multipolar world. Just as the International Atomic Energy Agency and the Non-Proliferation Treaty were established to manage civilian nuclear power without enabling weaponization by bad actors, he proposes that similar international frameworks are needed for AI to ensure open-weight models do not become weapons of mass destruction held hostage against humanity. He challenges the notion that China cannot be negotiated with on safety issues, citing historical examples like Nikita Khrushchev and the Cuban Missile Crisis to illustrate that even difficult adversaries can reach agreements when there is a clear shared interest in preventing disaster, urging policymakers to pursue dialogue rather than assuming an impossible barrier exists between Washington and Beijing. Regarding export controls on advanced chips announced in late 2022 with the hope of maintaining a significant strategic lead for American AI models, Mallaby admits that after three and a half years these measures have not delivered the anticipated overwhelming advantage over China's frontier models. Current studies suggest the United States is only about eight months ahead at best, a gap that likely shrinks further when considering how quickly models are converted into practical applications, meaning both nations now possess powerful AI capabilities regardless of chip restrictions. While he remains open to maintaining controls if future compute demands create new opportunities for leverage, his primary recommendation is to prioritize collaboration with Chinese counterparts on safety issues even if it requires loosening some export constraints, emphasizing that the potential benefit of joint efforts to prevent bad actors from exploiting AI outweighs the marginal strategic gains currently sought through isolationist policies.

Read the full video transcript

What have you learned about AI from your trip to China and thinking about China, speaking to Chinese people, whether they're technologists or otherwise? Like, what have you learned during or since that trip? >> Back in March, before my book was published in the US, I went to China because the Chinese are faster at everything, including publishing books. And uh my publisher brought me out there and basically, you know, took me around four cities, eight days, meeting with AI leaders both in academia and big companies like Huawei and Hikvision and Ant Group. And the thing which was surprising was the extent to which people brought up the issue of AI safety. And I say that was surprising because my friends who had done AI policy in the Biden administration had primed me to expect that there would be no mention of safety in China, that they basically didn't care about it, that you know, the muscle memory that we have in the West of technology being dangerous, you know, the atom bomb experience, the Cuban missile crisis, our ambivalence about technology is not shared in China, where their idea of catastrophe is sort of like, you know, the Cultural Revolution, it's some political thing that goes wrong. And conversely, technology has been part of their amazing growth story in the last 25 years, which they are rightly proud of and delighted by. So, they love technology, right? So, when the Biden team tried to meet with the Chinese and talk about AI safety, they got nowhere and they decided it was impossible to even talk to them about some sort of non-proliferation treaty for AI. But when I went there, I found they did talk about safety kind of unprompted. And this led me down this track of arguing over the last couple of months that the door is actually open to a dialogue with China about preventing bad guys doing bad stuff with AI. Because they don't want the internet to be crashed by some cyber hacker who has the tool. They don't want bio weapons, they don't want chemical weapons, they want none of that. They love regulating the internet, right? So we have a shared interest with the Chinese in preventing this proliferation of risk from going nuts. And as I thought about it, you know, the kind of Cold War analogy came to seem more and more opposite, right? So if you look back at the story of nuclear weapons, there were two kinds of danger. First danger is you have a nuclear war between the Soviet Union and the United States. But that was contained by balance. Two superpowers, they both have the weaponry. They have mutually assured destruction, so there's no war. Then there's another kind of risk, which is that other random rogues, whether it's criminals, terrorists, rogue states, get the stuff and they do bad stuff, and it's much harder to deter that because it's a multipolar game. And so deterrence doesn't work so elegantly. And so the way it was dealt with in the Cold War was that in 1956 there was the agreement on the International Atomic Energy Agency. And in 1968 the Non-Proliferation Treaty kind of enforced compliance with the IAEA such that you could get civilian nuclear power if you were a non-nuclear state, but you had to submit to the rules and be inspected and show that you were not using the enriched nuclear material to build a weapon, right? And so I think the same analogy could be applied to AI. We're going to have parity roughly with China. We'll both have powerful AI. Hopefully deterrence prevents war breaking out. But at the same time, we don't want open weight models that can be freely downloaded by anybody who wants to fall into the hands of criminals and terrorists who can then use it to hold us hostage. And we have a joint interest in that. And you know, when my friends from the Biden team or even from the current administration say, "Well, you can't talk to China about safety. They don't care." I say, "That's not true." And they say, "But it's really hard. They don't stick by their commitments." And I go, "You think Nikita Khrushchev in [snorts] the Soviet Union was easy to negotiate with? He was the guy who put missiles in Cuba and went to the UN and banged his shoe on the table and said, "We will bury you." I mean, he was a tough guy to talk to, but we did talk to him and we got the non-proliferation treaty agreed. And I think we need to do the same thing again now. >> Where do you stand on your thinking about chip export? >> So, when the chip export controls were announced, which was October of 2022, right before ChatGPT, I supported those controls quite loudly. I wrote a very long piece in the Washington Post saying that if we could stop China getting frontier models by depriving them of frontier chips, I was all in favor of that because of the strategic advantage for the US. I mean, I work at the Council on Foreign Relations. We do geopolitics and national security all day long. And I'm all in favor of US power. But I have to say that, you know, 3 and 1/2 years later, we haven't actually achieved that enormous advantage over China in terms of the models. Based on the best studies, we're kind of 8 months ahead in terms of where the frontier model is, like our frontier model versus their frontier model. And then if you adjust for the speed with which the model gets turned into an application, probably that gap shrinks, and it may even be nonexistent. So, however you slice that, the basic bottom line is we both have strong models, and the chip export controls have not delivered what I hoped would be the big advantage. And so, I'm not against keeping the controls on if we think that maybe as the compute demands of bigger and bigger models bite, the chip controls will bite more, and maybe we get a bigger advantage next year or something. But I don't want the chip controls to get in the way of discussion with the Chinese about where we have a shared interest, which is in controlling open weight models and preventing the bad stuff falling into the hands of the bad guys. I would prioritize collaboration with China, and if that meant, you know, loosening up a little bit on the export controls, I would be okay with that.