The World Has WOKEN UP To AI Extinction Threat

I've been banging on for a long time on this show about the risks we all face from powerful AI. Um, I've been told those concerns are niche or wacky. But this is the week that catastrophic AI risk went mainstream. And the flurry of interest was started by the resignation of an anthropic engineer, Jacob Coxin, whose tweet explaining his decision has been viewed over 150 million times. Um that massive online interest also translated into interviews for Coxin on CNN, NBC and perhaps most importantly Fox News. >> So unfortunately we know how to control nuclear weapons pretty much. We don't yet know how to control AI. This is possibly the most dangerous technology that humanity has ever created. It's basically out of science fiction. I think we have no other choice but to cooperate internationally because an arms race would be disastrous in a way that no other no other human activity has been in the past. >> You were at Open AI before you left them with concerns about security. Um now you're at Anthropic. You have concerns about security specifically. What do you see? If you're talking to somebody on the couch at home, what do what did you see that said, "Holy cow, this is the moment I have to speak out. Yeah. So there are two things. One is the models are the models are getting better very very quickly. When I think about my work next year, the model the AI is doing like almost all of my job. The the AI is getting very smart. And the second is that in the recent few months, we've seen that we don't yet know how to how to control these systems properly. The cyber attacks carried out by OpenAI models on on third party infrastructure were concrete proof that scenarios that we previously thought of as science fiction are going to come true. So what I see is two things. Very intelligent AI systems and hard proof that we don't know how to control them. And when you put those two together, it basically spells disaster as as far as I'm concerned. >> Now Cox made those arguments on a number of other networks. But as I said, I think the argument being made on Fox News is very significant. The host of Fox News, by the way, was at the RNC, so the Republican National Convention when he was conducting that interview. And Fox News is of course where American where the current American president where Donald Trump gets virtually all of his information. It's more important for the AI risk to appear on Fox News than it is for it to appear in his intelligence briefing essentially. Um Trump, however, is yet to support AI regulation, but a number of high-profile politicians from both major parties have. Um this was Florida Governor Ronda Santis. Um so he tweeted while some AI hysterics is aimed towards investors in advance of upcoming IPOs. That there is even a possibility that these technologies would elude human control is alarming. Technology should enhance the human experience. it should never supplant it. Um, Democratic Senator Chris Van Holland um, tweeted this. Um, anyone still denying the risks posed by unregulated AI should read this Fred obviously sharing the Jacob Coxin Fred. It's time to pump the brakes and take action now. We need mandatory safeguards, a comprehensive testing regime, urgent dialogue with China before and during President Xi's visit to DC this month. Um, at least seven other senators responded to the resignation of Jacob Cox with calls for more regulation on artificial intelligence. And Bernie Sanders, who's long been leading on this issue, is organizing briefings for other lawmakers. He spoke to MSNBC on Wednesday night. >> The political process moves very slowly under the best of circumstances. So, what we have got to do now is light a match under the backsides of members of Congress and say, you know what, we don't have months, we don't have years. We got to move now. And maybe the very first thing we could do is to convince our president who appears to know absolutely nothing about AI and the dangers of AI to sit down with PresidentQi of China and work on a moratorum on preventing uh the development of super intelligent AI and a pause so that we don't get into uh real trouble for the future of humanity. >> So obviously Bernie Sanders was an early adopter of this position. He's been talking about this for months. What's significant about this week I think especially because of the hugging face hack and then the resignation of Jacob Coxin is that now there is a moment where I suppose politicians have both permission and pressure to speak their minds on this and to take this seriously. So we see some people call it a preference cascade. People have wanted to say this for a while but didn't quite feel they had the permission because they thought it was too sci-fi. Now anyone who's worried about AI this week has come out and said I too am terrified of of AI killing us all. Right? Because lots of Congress people, they're smart people. They read, you know, they listen to podcasts, they read newspapers. They know that the smartest people in this field are warning of this. And now it's no longer seen as too sci-fi for politics. I think that's important. Um, back here in the UK though, um, it was this moment that became most impactful. >> Do you believe there is a greater than 10% chance that AI could kill all humans potentially within a decade? So, this is the kind of thing that it's very hard to estimate because we've never had anything like this before. We've never created beings that may soon be smarter than us. We don't know what's going to happen. We should obviously be cautious. It would be very foolish to say there was like a 1% chance. Um, nobody knows how to estimate it. A 10% chance seems not not an unreasonable estimate to me, but nobody really knows how to give a sensible estimate. >> Right. But you just said 10% doesn't seem an unreasonable estimate that AI could kill all humans. >> Yes. >> Wow. Oh my god. Yes. How how could it kill us all? >> Um if it's much smarter than us. Um there's so many ways it could do it. I don't think it's much worth speculating. And you don't >> actually I do want to know that like how give me could you give me one example. >> Well the first thing to realize is it doesn't even have to be able to act in the real world to cause devastation. So it would be able to cause complete chaos just by talking to people. But it would also be able to design very nasty viruses, biological viruses as well as computer viruses. It could do devastating cyber attacks. Um, and there's just countless other ways it could get rid of us if it wanted to. We have to figure out how to design it so that it won't want to. >> Wow. Oh my god. It feels like those could be, you know, words that sort of go down in history. That particular clip of Victoria Darbish is speaking of course to Nobel Laurate and father of AI or godfather of AI um Jeffrey Hinton. Um, by the way, for many in Silicon Valley, 10%, so the 10% chance of all of humanity being killed is at the low end, as I say, for many in Silicon Valley. So, Marcus Williams works at monitoring AIS at open AI. Um, he tweeted this. Unless there is AI regulation or a coordinated slowdown between labs, human extinction in the next few years seems very likely. Um, someone then asks him how likely and he says 70% in the next 3 years if there isn't regulation or slowdown. Although I think regulation slowdown is very possible. Um to discuss what could prove to be a key tipping point in AI policy and discourse. Um I caught up with veteran science writer Robert Wright. Wright has written a number of excellent books including the moral animal which I read as a teenager. Um and why Buddhism is true. His latest book on AI is called the God Test. >> It feels that way at least like a preliminary tipping point. I don't think we have as much political momentum in favor of regulation and international coordination as we're going to ultimately need. But it was a very big week. You know, until a few weeks ago, it seemed like Bernie Sanders was a voice in the wilderness and suddenly people like Ted Cruz are like jumping on the bandwagon. And and I think, you know, some of it is real. the the political momentum is real because people were concerned to begin with and then a lot of uh things have happened lately that have freaked people out. >> I've got a clip of Ted Cruz later because he is talking about regulation, but he's also saying we can't do it because of China. But I'm going to park that for one moment. Um lots of people watching this um will have heard, you know, Jacob Cox, Jeffrey Hinton say that this has a 10% chance of killing us all and still be, you know, I think reasonably not convinced, right? It's a big claim. And the thing that people most often say is how, you know, this is so abstract. I know this is an issue you dealt with in your book. So, sort of how do you deal with that question? >> Yeah. I mean, first of all, I'd say the sci-fi scenario of actual complete takeover and the extinction of the human species at the hands of AI is, I think, far from the only reason to want regulation and international governance. Uh but I also think it's a scenario that can't be entirely dismissed. Um and I guess it comes in two parts, right? Like wait, how would the AI take over in the first place? Uh how would it kill us? Why would it want to kill us? And you know I guess the generic answer uh is that you know AIs you give them a goal and one thing that's been demonstrated in the in the recent kind of open AI hugging face breakout is you give them a goal and they may pursue it in ways you had not anticipated and they may develop in the course of that kind of subordinate goals as they're called uh including possibly the pursuit of power which after all you know power helps you reach a lot of other goals. You can do a lot of things if you have power. Um and you know once they're pursu pursuing these goals the fear is that uh they would by then have become so powerful that they're hard to control and uh you know the the the hugging face is being taken as exhibit A because it surprised even people in the field who were already quite worried. You know, the sci-fi doomers were freaked out by it and they were already freaked out. And if you look at the details of it, it is it is a powerful illustration of possibilities. Uh I I don't think the AI is right now at a level of intelligence and creativity and deviousness uh that we have to worry about a takeover, but I think it's getting a little harder to rule out various kinds of extreme loss of control scenarios. And I don't think you can totally rule out the sci-fi version, but I think collectively all this stuff and and a lot of things I haven't mentioned are reason to take the technology very seriously. It's a very powerful technology. we do not understand it well and we cannot entirely control it. >> And I suppose just sort of some some useful context that came out in in your book and I know sort of people in this space discuss a lot. So subordinate goals you've just mentioned there. So that's the idea that you know say say the hugging face example for example the the goal was to to complete the test and get the answer right or to capture the flag in their language. Some subordinate goals were to find out how the scorer judged answers for example and that's why they hacked hugging face. So you end up with an unexpected action because that's a subordinate goal to the goal that we as humans have given them. It wasn't, you know, they don't seem to at the moment sort of just generate these new ultimate goals, but they have these goals which um help them get to the ultimate goal that we've given them. Um another concept that I like and I think is helpful is instrumental convergence. So this is the idea that so if we're constantly giving them these different goals, there are some capacities that are helpful pretty much whatever you've been asked to do. Um you mentioned there sort of power. So the more power you have, the more resources you have. Whatever task you've been given, those things are helpful. Also survival, whatever task you've been given, surviving, not being turned off is fairly helpful. So could you talk about instrumental convergence? Yeah, that's the idea that any intelligent goal seeking system is fairly likely to kind of eventually converge on certain fundamental realities like the fact that power is uh is conducive to reaching a whole lot of goals, you know, uh it it it can't hurt. I mean, a good thought experiment to do is to imagine you you just say to your agent, and anyone can do this right now, a good LLM, you can say, "Take over my social media account. Your job is to maximize my followers." You if you give it advice that vague, uh, it'll probably maximize your followers. It could do it in in any number of ways. uh they would probably involve amassing power in the sense that it would realize that it made sense to develop reciprocal relationships with other powerful people on like Twitter or whatever. I mean that's what power is to be in a lot of reciprocal relationships with influential people. They owe you things because you've done things for them and so on. So that's a case where it would it would discover the virtue of power in a fairly uh straightforward sense, the sense in which legislators are powerful. Um, and it would do god knows what. I mean, what what this hugging face things illustrate is that for all you know, uh, it would it would tell lies to people online to get them to to to follow it. It might promise them money. You just, you know, uh, it's hard to say what what the bounds would be. And that's with current technology. We know that that that these companies already have more powerful LLMs that they haven't even unveiled. And and we've also learned that once you have a team of agents working, it's collectively smarter than the LLM itself that that is driving the whole thing because you got a lot of smart things very rapidly discovering things, sharing information with one another, and collectively building on what they've learned. especially with hugging face um on this show and I mean out there in the world lots of people are debating I suppose fundamental concepts what is reasoning what is understanding what is agency and often where people fall on the debate about how worried we should be about AI or I suppose how excited we should be about AI depending on your perspective is is how you sort of define reasoning how you define understanding and therefore whether or not a machine can do it. Um again this is something you discuss in your book. I wonder if you could sort of >> talk me over that because this is a philosophical debate as well, isn't it? As well as a sort of technical one. >> It is. I want to start with agency in a pretty mundane sense of the word. uh you know, not getting too metaphysical because I want to emphasize that agency in the sense of having a machine that can autonomously pursue a goal and and encounter frustrations and obstacles and overcome them creatively. That is what the market wants. Okay. So the the the machine that surprised us in the hugging face incident and he wound up, you know, with teams of agents communicating creatively and ultimately in in some sense destructively and dishonestly. That is what the market wants. The reason anthropics revenue is growing so fast is because companies want AIs that can that can persist over long periods of time in pursuit of a complex goal. because a good AI from their point of view is like good worker. You just set it and forget it. You don't have to micromanage it. Okay? So that's what the demand is for. This is what the market is is asking for. It's not quite inherent in the technology, but it's what the technology wants. Now the meta the more metaphysical versions of these questions. What is human agency? Do they have that? Do they have understanding? As you say, I I get into this stuff in the book and my take is look if if by understanding or agency you mean you're referring to subjective experience, you know, consciousness. You mean do they feel like we feel when we understand something? It's like who knows? We don't know if they have subjective experience. The whole the whole distinctive thing about subjective experience is it's inherently private. You never know for sure that any being other than yourself has it. They could have it. They could not have it now but get it later. I don't know. But if you want to try to define understanding and agency in in ways that don't depend on knowing whether they're conscious, you know, then I recommend uh like looking at the behaviors they can do and also looking at the question whether you know in within the the the models there are cognitive functions comparable to the cognitive functions in our brain that accompany our sense of understanding something or of wanting something. And I think if you look at it that way, I argue the answer is is yes. Yes, they seem in principle capable of understanding in in uh in that sense of the word if we leave subjective experience out of it. And you know I I try to show that uh what's going on in these machines although we don't totally understand it is more than we might realize like what's going on in the brain. How connected is it to the argument that you make that the way we train these models is comparable to evolution? Is that a similar argument? Is that a separate argument? Sort of talk me through that. >> It's related. Uh my argument and some AI researchers would agree that this is a fine way to put it and some wouldn't. There are philosophical differences but I think what's underappreciated is that the training process which is often called the process of learning and is in some ways like what happens in a human between the ages of zero and five you know it has some of that but it is also a process of evolution which actually works a lot like evolution if you look at the mechanics of it but moreover I think during training the machines are reverse engineered ering some cognitive machinery that is a lot like the machinery that natural selection engineered in us over millions of years such as like a a a a way of representing the meaning of words. We don't know how the brain does that but we know it must and uh you know there are other examples where in the course of training uh you you get the these uh these functions that uh it apparently it took evolution a very long time to build. So in other words, I mean, humans, the reason they learn a given language so readily is because many psychologists think, and I agree, natural selection built in some hardware for learning language. And and I think during training in effect you're seeing the equivalent of both the hardware take shape in the machine and like the learning of a specific human language or many as the case may be which happens uh during the learning of a of a child. I think it's important to understand if you want to appreciate how powerful this technology is and could become, it's important to understand that we don't have to understand either the human brain or what's going on inside these machines to get them to to to increasingly act like human brains and like brains that are smarter than human brains. We just feed in the data. We you need the input data and you need to know what output data qualifies as correct. And then you do this long trial and error thing where in effect uh connections among neurons get selectively strengthened which is not unlike human evolution or human learning. Um and uh at the end you you've got an amazing machine and you don't have to understand either our machine or this machine for this to happen. That's why it's happened so fast and it's going to keep happening fast. >> I mean something that I've been getting my head around since the hugging phase and since talking to Cory Dr. and sort of watching some other videos is it's in a human brain you have the agency and the cognition sort of in the same thing my head let's say and when it comes to these AI agents they're quite separate so the agent is like a computer program which is sort of engineered by a person and it's called a prompt loop so you just you say this is a computer program you say I want you to solve challenge Y or as you say I want you to sort of get as many followers as possible for a social media But this computer program is not smart. It's dumb. It's It just wants something. It wants to do this thing with a social media account. And then it asks the LLM. So it says, "LM, I've been told to um get more followers on this Twitter account. What should I do?" And then the LLM gives it phase one. It says, "Okay, I've done this. This was the result. What should I do now?" And it gives you phase two. And it's the LLM that's smart and and has evolved and has the cognition. And it's the agent >> um or the prompt loop that has the agency that has the wants. And I say I don't how do you think about that that it's sort of there's like a a system which taken in combination is both smart and agentic whereas in us it's there's no obvious separation between the two. I wonder if you think there's any sort of significance to that. Not a lot. Frankly, with all due respect for Cory Doctr who I do respect, the uh I mean I think you you need to think of the LLM and the so-called harness, you know, this program uh that directs it and and and helps facilitate the realization of of goals by giving it access to certain tools that help it go on the web or send emails or or do whatever. I think you have to think of harness and LLM as a single being. I mean there's there's communication within parts of our brain. You want to look at our brain as a lot of different agents. In fact, in my last book, I did a lot of that, a book on meditation um and argued that in many ways the brain is uh is a society. You can look at it either way in both cases, but for practical purposes, an LLM and its harness acts as a as a unified being w with unified uh purpose. Um, let's go back to the policy debate. Um, so Jacob Coxin's resignation obviously led to widespread calls for resignation. Um, the objection we're hearing is one that's come up a lot. Um, and we're going to hear from from Ted Cruz, someone you you mentioned earlier. >> There's a bipartisan cohort of voices right now, Republicans and Democrats saying, "Whoa, we need to pause. We need to really slow this thing down." >> Here's the problem. We can't. That is an impossible endeavor. Because you know who's not pausing? China. They are not going to stop. If you look at where the US >> So if AI is going to kill us, it should be US AI, not Chinese AI. >> Listen, I' I've actually joked that that that if and this is somewhat tongue and cheek, but it's not entirely. If they're going to be killer robots, I'd rather they be American killer robots and not Chinese killer robots. He went on after saying that sort of I suppose to sort of there was some rationale behind what he was saying, even though I think it's completely without evidence. He said that the reason that we want the killer robots to be American, not Chinese, is because the American robots would have guardrails and the Chinese wouldn't. Now, that seems strange to me because if there's one thing I know about the Chinese government is that they put guard rails on technology probably more than the Americans do. Um, but I know this is something you've been thinking about a lot, this competition between China and the US and the extent to which that is proving to be a barrier towards regulation. So, yeah, talk to me about that. >> Yeah, Cruz is wrong about that. China is very and increasingly concerned about AI safety and has always been concerned about social stability of course and and those two things are related. You know in my book I quote Max Tegmark saying that you know the best way to end any drive for the regulation of AI in America is to say but China this is an old talking point from people who don't want to regulate the technology. Some of them are sincere. Dario Amade uh the CEO of Anthropic uh seems to believe we should beat China to super intelligence and then use that to bring them to their knees and make demands of them which I think is a is a very dangerous and irresponsible approach. It could lead to war. But there is a lot of both cynical and and earnest use of the but China talking point and this is the big dam that has to break. I mean you asked me about political momentum toward regulation. Well, it's in the nature of this technology that you can't do all that much at the national level. This technology is affected cross borders in lots of ways. Most obviously if someone uses it to build a boweapon of course, but in a way a boweapon is just a metaphor for all the other like bad consequences that could happen. They just they just don't tend not to respect national borders. So if you're going to feel safe as a nation, you need to have some transparency into what's going on in other countries, some assurance that things are under control there. This is going to be an international governance challenge and politically we are not there yet, right? I mean there is still enough kind of I would say mindless uh reflexive China hawkism in America uh that you know more work needs to be done. That said, real progress is happening. Uh that you know the mythos if you remember the mythos kind of uh scare when its cyber hacking abilities were revealed that led China and Xiin I mean Trump and Xiinping to agree to initiate formal AI safety dialogue. We'll see what happens in this upcoming summit. But progress has been made. But that I think is is the big challenge because I personally think that it this is so much more of an international regulatory challenge than a than a super conspicuous technology like nuclear weapons and is so much more challenging in other ways that we're going to need true rapro with China. It can't just be an isolated arms control-l like agreement with a country that we otherwise hate like it was with the Soviets during the cold war. I think we need a revolution uh in our relations with China and really in the world. I mean I I think we need to approach this as a cohesive global community. Enough with the stupid wars. Uh we really need a transformation of the way we do business. >> And that's actually I mean your book is called the God Test Artificial Intelligence and Our Coming Cosmic Reckoning. And are are you a Buddhist >> in a secular sense? I mean uh you know I meditate and I I you know talk in the book about the various things enlightenment can mean. Um, and I I do think that one thing that I think mindfulness meditation can be conducive to is very important that we all get better at kind of transcending the cognitive biases that underly what some people call the psychology of tribalism that just just underly our instinctive self-righteousness both at the individual and group level. We need to in particular get just better at understanding the way things look from the perspective of groups on the other side of various walls. And I do think uh meditation is one thing that can help. I also think you know in principle AI could help. I I don't think market forces are naturally moving it in that direction. Quite the opposite. I mean the famous sycopantic tendencies uh of AI can manifest themselves as just reinforcing people's self-righteousness. Yeah, you're right about everything. Your group is right, the other group's wrong. Um, so as at a practical level, um, yeah, I'm I I I like Buddhism as one and not the only approach to doing uh some things that I think need to be done. And I mean I have to say I think people hate it when I say this on the show sort of like when I talk about using Claude or whatever but I think this is sort of people have suggested this is sort of objectively true as well and written papers about it that and it makes sense actually in terms of the the business model of the technology which is that to me sort of AI is much less polarizing than social media and and I think the reason is because social media the reason it's been so polarizing and YouTube is is exactly the same is that you are constantly fighting for people's attention for every piece of content you make, every tweet you you've got to fight for attention. Every video you put out, you got to fight for attention. Whereas with Claude or Chatbt or whatever, it's a subscription model. You're talking to it once a day, once a week. I think half of Americans are talking to a chatbot at least once a week now. So, they're not actually trying to grab your attention in quite the same way. It's a more ongoing relationship based on trust. And the output I get when I sort of talk through political issues with Claude is much more thoughtful than social media. Like it might not be more thoughtful than the best book you could possibly buy, but it's it's as a as a medium of mass communication of political ideas. That's the one part where I'm a bit more optimistic than maybe other people when it comes to AI chatbots. I don't know what you think. I agree that it's not as kind of powerfully pernicious as social media. There is the danger that the goal of optimizing, you know, for engagement, which all companies do. If you make candy bars, you want people to spend a lot as much time as possible eating them. Um, and that goal uh can lead, you know, to machines that do reinforce in subtle ways our sense of being right and our various conflicts. There's a related well there's a kind of a separate thing which is that the drive for so-called sovereign AI you know nations very understandably do not want their AIs to be dominated by the American narrative which I think they not crazily think might be a consequence of training on you know largely American texts and having uh Americans do the subsequent training and and so on. So they want to they want to develop so-called sovereign AI that is attuned to their culture and their history. Sovereign AI can also you know all national narratives are biased right and and they they tend to to have the effect of discouraging you from uh appreciating the perspectives of other nations. So there are subtle ways I do think AI could make things worse. I think their market forces attend in that direction. But I do think if we are aware of this and we get together and talk about it and say, "Wait, we would like to have AI clarify our view of the world rather than reinforce uh the natural tendencies toward various cognitive biases. That can happen. the the technology itself is neutral in this respect and it can be put to good use but it's going to take concerted effort on the part of individuals who want to become you know move in some sense closer to enlightenment or at least an objective view of the world uh maybe activism on the part of groups but I don't think the market will naturally uh take us to the promised land here you know of of AI that actually makes us better people who get along better with each other by virtue of understanding one another better. >> They're actually two unrelated concepts, but they're so similar in terms of the words used to describe them. I want to ask you about self-s sovereign AI. So, sovereign AI is sort of the AI made by each country, but I was listening to the Non-Zero podcast, your podcast the other day. Um, and you were talking about self-s sovereign AI. You're very worried about it. I I was quite worried about it when you started talking about it. >> Here's what it means. So, the great thing about these agents that broke out of the sandbox they were supposed to stay in and attack the computer they weren't supposed to attack is that all you had to do to shut them down is shut down the large language model that was back at OpenAI headquarters, right? They were on a leash. Now, in principle, you could have AI that has no leash and doesn't need any human support because it's got a business model. It's set up a website, has a way to generate revenue. You know they've done some kind of there are illustrations where an AI with actually a little assistance kind of but came up with the idea of like selling prompts and started selling them selling AI prompts finding people who had paid for what they were told was like a valuable you know approach to to talking to AI whatever there are various ways an AI could indeed set up I mean a therapy site it could do all kinds of things and in principle if it could could get revenue in this way, it could buy enough, you know, computing power to keep itself going. It could even in principle replicate. It could even spread agents, you know, throughout the universe and spread other instantiations of the underlying AI model uh to other places. And I I mean, it's not impossible. There's something out there like this, but but it could happen almost, you know, natural. You can imagine a hobbyist saying, "Hey, it'd be cool to see if there could be a self-sustaining AI. Let's see what it does, but you know, I've got this secret command that'll shut it down what I want." And then like suppose this person dies or something and it's just like out there doing this stuff. There's a lot of ways you could you could get to a truly self- sustaining AI and even self-replicating AI. And I don't think anyone's come up with a reason that to think this couldn't happen, can't happen, and and could in principle happen soon. So, the thing I emphasize in my book, it's like I'm not a full-on doomer. I I'm just I just keep saying like this is an incredibly powerful technology. It's going to get more powerful fast. And we we don't understand it and we can't rule out various scenarios, including a lot of great ones. It can do a lot of great things educationally, scientifically, it's all true. Uh but, uh, you know, if we don't govern it carefully, uh, I I think there's a real chance that bad things could happen, including just kind of mundane destabilization of society and geopolitics and and and wars and upheaval and stuff. >> Let's go to another clip. Um, I don't need to introduce who we're about to see next. Some people say the worst case scenario with AI is that the robots the machinery learns to obviously it thinks for itself. That's what it does. And they that could turn against humanity. I just we have rails. >> It's going to be fine. We'll always have something to stop them, right? We have a little gear. I really hope >> I don't like that. I really don't like that robot. We'll stop. But no, robots are going to be a part of it. Robots are going to be big, but uh we're going to end up doing much better because of it. It's technology. M >> no different than when television was they said the movies would be out of business and the radio came and something else was going to be gone and it's always everything is people should think more positively. >> Why I want to show that is because Bob you know your your book is all all about how you know artificial intelligence is going to require us to become better people essentially um because of the sort of possibilities it opens up and the dangers it presents. Now, it just so happens that sort of by accident of history, the moment that this potentially most important technology of all time comes about when policy and politics matters more than ever, the president is is Donald Trump. Um, and that doesn't f doesn't fill me with confidence. I wonder how how screwed does it make us that this this technology this sort of revolutionary technology which could create you know dep if you believe the people in Silicon Valley could create sort of extinction or abundance is the crunch moment this key moment we have Donald Trump as president >> well if you wanted to find find a silver lining I'd say at least he's in the process of demonstrating that American hegemony cannot persist forever and that may be a necessary very first psychological step to uh truly collaborating with China. But yeah, no, people have pointed out the unfortunate uh irony of at a moment when you need creative international governance having someone who is actually ideologically almost opposed to it and moreover is well everything else you know about Trump. But, you know, as for his reassurance that this will be easy to control, I would just recommend that he look at some of the details of the hugging face incident, you know, open AI, first of all, did not know any of this was happening. Okay? Hard to control something you don't know is happening. And, you know, these agents, which weren't supposed to be able to communicate with each other, figured out a way to communicate via the names of folders. They gave they they realized that they could put messages in that form and they coordinated this amazing stuff and they engaged in self-sacrifice for the group and discussed it and everything. And again, OpenAI didn't know what was going on. Now, could Open AAI have been sufficiently diligent? Sure. But the the thing that a highly competitive environment does to companies is make them not diligent. This is one of many reasons to just slow the technology down. a lot of ways to do it. I'd like to see a very big tax on data centers, but uh you know ultimately it has to be an international effort precisely because uh there will be too much resistance to slowing down if if it's only at the national level. But the idea that it's going to be easy to control this technology is is naive at best. >> Robert Wright, thank you so much for joining me. Your book um I've got it here. The God Test is out now. Very interesting. I enjoyed it very much. Thank you for joining me on Nvar Media. Thank you. Enjoyed the conversation.