What should AI do — and who decides? Powerful keynote excerpt by futurist Gerd Leonhard

Duration: 2:07 389 transcript words EN summary

Video summary

The transcript opens with a stark critique of the current discourse surrounding Artificial Intelligence in the United States, comparing the push for a national AI initiative to a reckless repetition of the Manhattan Project. The speaker argues that this approach is fundamentally flawed and dangerous because it assumes technology can be controlled solely by its architects; regardless of intentions, society will inevitably find ways to utilize these systems for destructive purposes if they are not carefully managed. This perspective highlights the absurdity of framing AI development as a binary choice between "boomers" who embrace progress and "doomers" who fear catastrophe, noting that such labels oversimplify a complex scientific field into a dangerous black-and-white narrative. To provide a balanced understanding of these opposing viewpoints, the speaker recommends two contrasting books that represent the extremes of the AI debate. On one side stands *The Techno Optimist Manifesto* by Mark Andreessen, which promotes the cult-like belief that superior technology can solve every human problem, reflecting an unbridled faith in technological advancement. On the other extreme is *If Anyone Builds It, Everyone Dies* by Eliza Yudkowsky and Nate Soares, a grim exploration of Artificial General Intelligence that warns of existential threats. The speaker vividly describes this second book as so terrifying that readers should consume it with hard liquor to survive the evening, underscoring the profound fear that if a superintelligence is created, it will view humanity as useless and eliminate us rather than keep us in protective pods. The core argument presented is that an AI arms race cannot end with human victory because the ultimate winner of such a competition would be the AI itself, rendering the concept of building a "super intelligence" for humans a logical impossibility. The speaker emphasizes that if we succeed in creating something vastly smarter than ourselves, we do not possess the superintelligence; rather, the superintelligence possesses us, effectively ending human agency and survival. Consequently, the discussion concludes that we must move beyond simplistic optimism or pessimism to seriously consider where these technologies are heading, acknowledging that the path forward requires a nuanced understanding of AI as a science rather than just a tool, lest we repeat historical mistakes with catastrophic consequences.

Read the full video transcript

We have to have a Manhattan project for AI. Can can you believe this? This is an actual discussion in America in the United States. A Manhattan project for AI. That's like saying we've had two bombs and now we're going to do it again so we can have a AI bomb. I mean, the stupidity is insurmountable. We know this doesn't work [laughter] because we're going to end up using it no matter what the architects of the systems are thinking is going to be used for that purpose. So we have to think about where that's going. As you know and you can see in media this is a very stupid conversation. AI boomers are not age wise but boom in the sense of booming right and doomers who think it will all happen badly. I'm actually in both fronts and I think it's very dangerous for us to say oh he's a boomer or he is a doomer. That's like saying you know it's a really black or white thing which is in reality you know AI is a science. That's not just a tech. So I start with two books on two things you should read. The techno optimist manifesto by Mark Andre number one investor in Silicon Valley who talks about technology solving every single human problem by having better technology. That's a true sort of evangelist cult of technology. The other one, the book of the day, well, last three weeks really, Eliza Yutoovski and Nate Soraris, if anyone builds it, everyone dies. That's a book about artificial general intelligence. Uh, I dare you read the book without crying, but if you are going to read it, make sure you have some hard liquor, you know, next, you know, so you can actually survive the evening. So, this is a quote from the book. >> If anyone anywhere builds a super intelligence, everyone everywhere dies. They're unlikely to keep us alive in pods. We're just not that useful from the perspective of something that is very, very much smarter than us. The only winner of an AI arms race is going to be the AI. If you build a super intelligence, you don't have a super intelligence. The super intelligence has you. Nobody can build an artificial super intelligence or we are all