The REAL REASON They Won’t Slow A I

When we're critiquing the AI industry, we normally focus on Elon Musk, Sam Alman, and Dario Amade as the power- hungry men taking us to the brink of catastrophe. Um, I think it's well warranted to focus on those three. However, I think there might be an even bigger villain in this story who we talk about less. Jensen Huang. Um he's the CEO of Nvidia, which is the company that designs the chips that power data centers and which thanks to the AI boom has become the world's most valuable company. Um he has the strongest interest of anyone in the data center buildout continuing at pace. Um which perhaps explains why as calls for an AI slowdown are growing, he's been the loudest opponent of any new regulation. And that's put him on the opposite side of scientists like Jeffrey Hinton and Yoshua Benjio, but it's put him on the same side as Donald Trump. And the cozy relationship between the world's most powerful CEO and the US president was on full display this week when Trump called Wang in the middle of a live podcast show. >> The great thing about life is that Jensen can develop the most complex computer chip in the world that nobody can copy for 10 years, but he can't figure out how to put ME ON SPEAKER. [laughter] THING WE HAVE TO REMEMBER this one. So just with the AI it's almost this conspiracy and the happiest group is China and China is very happy and I could even say in the country a lot of states are happy that weren't going to get anything because they're being uh inundated by people that want to be there. But now all of a sudden you see they're building in Finland. They want to build one. Google wants to build a big one in Finland, which I'm not happy about because they were unable to get permitting. And I'm telling you, it's all a hoax. The data centers are great and they make people wealthy and they make states wealthy and it's the oil of the next 20 25 years. It's bigger than the internet and the AI, you know, much more so. and uh they're just playing right into the hands of a lot of people that don't want to see it happen. And that could be political people and it could also be China. And we're not going to let that happen. It's a it's a hoax. And >> you're right. We're not going to let that happen, sir. >> No, we're not going to let it happen. The uh the robots are not going to be taking over the world and that's not going to happen. >> We're not going to let that happen, sir. Um Jensen Wang has you know the evidence of him opposing regulation isn't just that clip. He's been very explicit. He's saying we don't need any new regulation on AI. The companies if they're making anything dangerous they should just not release it. So he's saying it should all be left to the companies dismissing the idea that there is any kind of dangerous race or collective action problem and and basically saying you know let's keep this going. Now there's a good reason why he wants to keep this going right why he's very opposed to regulation. In the latest threemonth reporting period, Nvidia's profit doubled to $60 billion. So that's in three months. And for comparison, just three years ago, Nvidia's quarterly profit was $6 billion. So the AI boom has increased its profit by a factor of 10. This is not normal, right? Their profit margin is also enormous at 66%. So for every chip they sell, they make a 66% profit margin. That's because they don't really have any competitors. So Trump's current position on AI regulation will be and has been music to his ears. Um so on the same day he sort of had that impromptu conversation with Jensen Hang. Donald Trump posted this. I am the hoax buster and I'm right now breaking another hoax that AI is going to take over consume and destroy the world and that robots will be marching into our cities and getting rid of us all. This is even wilder than the Russia Russia Russia hoax or the global warming scam. Thank you for your attention [laughter] to this matter. President Donald Trump. Um >> Aaron, I thought this was interesting because I suppose it's because he's not as like weird or maybe duplicitous as like Dario Amade or um Sam Arman or Elon Musk. Maybe not so openly sort of political that he doesn't get quite so much hate, but he is the guy who is profiting the most from the AI boom, most directly from the data center boom. Um he at the moment is the closest one to Donald Trump. >> Yeah. Everyone seems to give him a bit of a pass like he's not public enemy number one. >> Yeah. >> Is he getting off lightly? >> I think two reasons for that. First he had a proper job, right? So when he was a kid, he grew up in the United States, but he worked at Denny's. This is this is a true story. Um I think was it Steven Wit wrote a great book about Nvidia and Yensen Huang FT book of the year last year. Really well written, fascinating book. He had a real job is one thing. And he's said things in the past such as, you know, there's um the potential wealth tax. There's the referendum on that in or the vote on that I think in November, right, in California. And I think that would cost him billions of dollars if there's a wealth tax. And his position is if people vote for it, I'll pay it. I'm not leaving California, which you know is very different some Elon Musk. So he's got some credit in the bank for for stuff over the years. But you're right, structurally he is benefiting more than almost anybody else um with regards to to this technology. Uh Nvidia, I think at one point passed $5 trillion in terms of its market valuation. And of course, this is the company that designs the chips that AILM are all using for the most part. I think about 25% of the market at the top end is like Samsung and TSMC. Um Samsung and uh >> Huawei. >> Huawei. Yeah. Sorry. The the but Huawei have their own sort of Chinese little acronym. Um but ultimately Nvidia is making almost all of these chips or designing almost all these chips and then of course fabricated in Taiwan by TSMC. Um [snorts] so I think he's got off lightly for for reasons understandable reasons. Um but it's an interesting one and I also say that for him it's a very dangerous game to be tying yourself to Donald Trump now. I mean the guy is going to have two years left after November. I mean, unless they change the constitution and there's still a big open question. Is there MAGA after Trump? Uh, I kind of feel like both political parties in the US are in this moment of transformation and I I could be wrong. Famous last words. I feel that the Republicans are kind of coming to the end of their transformation a little bit and the Democrats are just starting. Uh we're kind of seeing that with the first story tonight about Ed Sheeran and the changing sort of political views of of younger and middle-aged Americans sitting on something like Israel, right? I think it's a dangerous game for him to be playing to tie himself so closely to Trump now. Um however, you know, it may be the same reason why he's saying he's perfectly happy to pay higher taxes in California. You're the person in charge. I'll defer to you. And that's just that's just how he does business. It might not be particularly ideological. Whereas with somebody like Musk, it was heavily heavily ideological. Um, but also Trump Trump's position about hoaxes and things. My goodness, you know, uh, Jensen twang by all accounts of all of these people is the most intelligent of all these, you know, multi- multi-entillionaires. Um, centiionaires is the is the technical term, isn't it? And in the case of Muskat trillionaire, he is by all accounts the most intelligent. So, and you're and you're saying we'll stop it, sir, to a guy saying that it's a hoax and comparing it to, you know, climate change is a hoax and co is a hoax. >> Seems kind of myopic. >> I was going to say people prefer people who are doing physical manufacturing because it's real and they think that's a real job, but actually Elon does that as well. So, so it must be that Jensen Wang is yeah, he's he seems less ideological than the others, even though I think he's probably the one who is most driven by his bank balance, whereas the other ones seem to have other weird quirky interests. Um, Sam, I mean, Sam Alman, I think, also wants to be president of the world in a way that Yensen Wang doesn't. Um, meanwhile, anyway, let's go sort of what else we're learning about this story this week. Um, because more and more employees at the big AI companies have been going public with their concerns about the technology. Um Daniel Selum who works at OpenAI published a viral essay this week about two million reads I think that's one's got um so he writes I have been working on AI for over 15 years across many different paradigms I did early work on probabilistic programming languages at MIT was one of the early developers on the lean fear improver at Microsoft research demonstrated one of the first instances of neural network learning um to reason for my PhD at Stanford and since joining open AI almost 5 years ago have helped pioneer chain of for optimization on language anguage models and more recently data efficient pre-training methods. Like many others, I have become extremely concerned about how far language models have come and the risk that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third party oversight and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation and that merely pacing the frontier more carefully will not adequately limit the long-term risk. The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in context where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans. And what we already know about this is alarming. Models will increasingly seem aligned even when they are not. Um, so he's saying they're these experiments that they do on these models are going to become impossible because the models now know they're being watched. And I think he's he's in charge of watching them. So again, it's one of those ones where the person closest to the technology is saying we're all going to be screwed. He ends the statement by saying this. I want the glorious renaissance future as much as anyone. So he's one of those people who thinks that AI is going to give us I mean something akin to fully automate luxury communism. I suppose I have worked for it however torturously my whole career. It breaks my heart to see the potential in sight and forgo it. The argument that if we get there by growing models rather than engineering them, we will lose everything in the end seems very strong to me. I am still wrestling with it and its staggering implications. I do not have answers. But as a first step, I wanted to share my present concerns. So that's gone viral because it's someone currently working at OpenAI. Now lots of people saying why are these employees allowed to tweet this? Like have they been given permission by their CEOs because the CEOs want to hype the technology? um why haven't they signed an NDA? Um now Nate Suarez who we've had on this show he made an interesting point on breaking points um where he was suggesting that the statements from the AI companies are actually I mean in a way they're to try and for properly serious regulation but they're also to try and quell possible workers results revolts sorry workers revolts because lots of these people in these companies think this threat is is serious as you're seeing from this guy and so they're trying to sort of appease them with these statements and that's also why they're allowed to speak publicly because otherwise they get poached by one of the other companies. They got lots of power, these workers. Um, and when it comes to these workers, they are an interesting bunch of people. Um, and the Trump administration seems to on one level understand that. So, the AI companies are famously full of people from the effective altruist movement. Um, and the Department of Defense uh this week tweeted effective altruists in shambles. Now, it's not very often um that you get uh I mean, this is the Pentagon, right? tweeting about what was once a very I suppose niche philosophical movement coming out of Oxford University. All these quite like nerdy kids who'd read too much Peter Singer. Now you've got the state well it's not the state department it's the Pentagon sort of saying haha you know you've never had before them sort of arguing against cans or any other niche philosophical school but that's what's going on. Um so again very interestingly the effect of altruists and their cotery have realized they're potentially at risk of undermining the AI safety movement if Donald Trump decides that all of this concern is just coming from this particular ideological sect which he thinks is not aligned with his own interests and politics. Um so this is really interesting because Jacob Coxen so he was the guy who resigned from anthropic. um he appeared on Fox News and I think this was sort of strategically decided um that he wanted to point out that Trump and the effective altruists might actually have something in common. >> And I do I agree with with President Trump's tweet that global warming doomsday scenarios were overplayed. Like it's a very slow process and talking about doomsday is completely absurd. I mean, I also agree that um many other catastrophic panics in the past have been just fear-mongering. The reason this is this is unique is AI is moving so incredibly quickly. We we won't have time to respond to it before it's too late. >> So, that's very interesting cuz obviously Jacob Cotton keeps going on Fox News, right? Because he seems to want to communicate with Donald Trump and he's recognized that Donald Trump thinks these are just a bunch of like lefty people who I shouldn't trust. They've done all these other hoaxes. ya da and then he's going Fox News to say actually I don't think climate change is an existential risk. I think these you know I'm not the I'm not your bog standard lefty. We're a interesting group of people who are difficult to place ideologically. Obviously it's pissed off a lot of people on the left. What did you make of >> Yeah, I don't think Trump's going to have that much space in his brain for nuance is he? But firstly climate change is an existential risk. Although I suppose his point there is the timelines are very different, right? Um, of course there'll be lots of extreme weather events and disasters we'll see over the next few years and decades before we see something really terminal and awful towards the latter half maybe of the century or maybe after the middle of the century. His point though is that this could happen in just a couple of years, a handful of years. It may be too late to pull back. Um, my personal view is that the biggest exential risk to humanity today is is climate change. But I mean that's a coherent argument, right? And I think you're right. He's going on Fox just to talk directly to to Trump because he knows that Trump at, you know, 1:00 a.m. is sitting there in the White House watching Fox eating his uh his Cheetos, having his uh ultralarge, his venty Coca-Cola. So, I mean, it's a smart strategy and whether or not he believes what he's saying is of course another matter. I should say, um, yesterday I spoke to Daron Asamoglu, who is, I think, probably the world's top economist. If you go on the rep sort of league table of economists, he's number one. that downstream's out on Sunday. And he, you know, he's pro AI. He says, "I'm pro worker AI." And this idea that AI can only be LLMs and it can only be um an artificial intelligence of the kind that you're getting with anthropic or open AI and it's going to be a monopoly and it's going to swallow up loads of work and loads of value from other industries. Yeah, it can do that, but it doesn't have to do that. And I said to him, or I asked him rather, what could be the future of the UK economy? You know, we've had stagnant productivity for 18 years. He said, "You should be you should be going all in on pro worker AI. How can artificial intelligence help productivity for people who are already in work rather than destroying those jobs? How can you make those people immeasurably more productive?" I think AI can do that with electricians, with pharmacists, with nurses. He said, "We haven't even thought in these terms because these guys, Sam Alolman, Yansen Hang, Elon Musk have basically cornered just the realms of possibility when it comes to this technology." And I kind of agree and that also interestingly does. So if you take the existential risk seriously, which I do, that that gets around quite a lot of that because the risk comes from agents. Yeah. And the reason that it it makes sense, the reason the AI companies at the moment are investing in LLMs plus agents of working together as a unit is because they're [clears throat] specifically selling a story to businesses, which is you can replace workers with this. And you can't replace workers with a chatbot, right? You can't replace workers with something that helps a user that's sort of reading the answers. You can only replace workers with agents. It's agents that, you know, the whole paperclip example, it's agents that potentially threaten to cause chaos, as was the case with the hugging face incident. So, if you sort of move away from agents and towards a very good sort of helpful advice bot, which is what sort of these LLMs could be, then we'd be in a much better position. Um, just some background on what Jacob Coxin believes there, cuz I want to talk sort of more about this on the show cuz it's so interesting because we always, you know, we're often talking on this left right spectrum on this show. I think most people watching it, they know what sort of like the leftwing people believe. They know what the right-wing people believe and they know what the centrists believe. But in Silicon Valley right now, we have this section of people who believe a really specific thing which is really different to what everyone else believes and kind of explains what Jacob Coxin is talking about there because often they're long-termists. And what a long termist is is that there's someone you you spoke to um what's his face? you interviewed him. Um, the Scottish philosopher >> William Macascal. >> William Macascal. He he wrote the book on this. So, long-termists believe >> that the biggest threat we face or the biggest tragedy would be if there were human extinction because they care just as much about people in a billion years as they do now. And they say if sort of 7 billion people were killed but a billion survived, they'd still be in a position to create a new fantastic civilization. So, it'd be sad, but it's not that big a deal. If everyone dies, then that ends civilization and we've let down our future generations. And so a 1% risk of everyone dying is a much more serious problem than like a 50% risk of 7 billion people dying. So it's not it's not an FX share, but that's why they don't think climate change is existential because they don't think climate change is going to kill everyone. They think it might kill a fair few people, but AI is the only thing that might plausibly kill literally everyone. So that's the one that fortoalls our future potential. And then the other thing I think which explains what Jacob Coxin is saying there. I mean I'm sort of putting words into his mouth. I don't know if he is a longtermist but that's why lots of these people say this about climate change when compared to AI. The other thing is they are tech optimist. So they basically think that you know humans are very inventive when we've had problems before. We've invented a technology to solve it. We probably will invent a technology to solve climate change. We've got enough time to do it. But AI is different because you can't create a technology to solve AI because AI is the technology. you've handed over the the role of best innovator in the world to a different species and you can't out innovate them because whatever innovation you come up with to get control over them again they'll just come up with a better one. So that's sort of how how if you're watching that and think what the hell is that guy talking about that's the intellectual world these people are coming from. >> Yeah. Uh and if you if you disagree with that sort of proposition about climate change just watch my video with Bill Maguire. uh climate change isn't going to kill everybody so to speak, but it will in the long term create potentially certainly for primates an uninhabitable planet. So I suppose they would say well we can just you know colonize other planets and we can terraform them or we can sort of retraform this planet. I think if your argument to save humanity is based on tech technologies which don't already exist. That's that's quite shaky. That's quite shaky as a basis to make that argument. Like Bill said, you know, it's very possible that we create a cascade of feedbacks in the next couple of decades which lead to a planet of four, five, six degrees warming. It will simply not be habitable for um for human beings. >> Bill Magcguire's take is also controversial. Obviously, the the effective altruists with their X-risk take that's very controversial, but the sort of central estimate of climate scientists at the moment is we're heading for something like 2.5 degrees by the end of the century. So obviously I think people should take Bill McGuire's >> I don't think that's true anymore. Michael, >> seriously, but the central estimate of say like I don't see is about 2.7. I think >> even the guy that you were speak What was his name? The other the other guy you were speaking to house father. >> Yeah. Even he was saying that with El Nino next year, we're going to be looking at potentially 1.7 above industrial pre-industrial levels. Right. So when people talk about two degrees warming, they mean two degrees warmer than I think the pre-1850s. He's saying that with El Nino next year, this is next year, we'll hit 1.7. Right? We're already at 1.5. And people say, well, 1.5 to stay alive already hit 1.5. The point is you have to do it over 11 out of 20 years. That's how it's being measured, right? So really, we've already gone past 1.5. Elino takes us to 1.7. What he said to you, Michael, is that that number is going up by 0.2 degrees every decade. Okay. In other words, in 20 years from now, that baseline will go up from 1.5 to 1.9. And when you get Elinos, it's already gone past 2.5. And that's in 20 years. So I I just don't think that's um it's a middle estimate, but I I just don't think it's a sensible one. [snorts] >> I'm just trying to I'm just trying to So if it's 0.2 per decade, then by the end of the century that would be what's 8* 0.2 >> you would presumably say there's going to be some sort of this some sort of ceiling on it or decelerate. But right now that's where we are. I mean so if that's if that's the case then >> they're looking at the trajectory of electric >> by the mid yeah but by the mid 21st century we've gone beyond we've gone beyond two degrees quite comfortably >> but the central estimate is 2.7 right it's not >> well I saying by the mid by the middle of the century not by the end of the century and then you incorporate elinos etc and then it's very easy to foresee that by the '60s 2070s we're already way past 2.5 with El Nino years this is it's kind of crazy to play this down to me when the guy you were talking to who's not a radical is saying it's going up by his central estimate by >> 20 100 it wouldn't look like Bill McGuire's like I've read his work he's not >> but there's a tension there's essentially saying two separate things he's saying it's going up by 0.2 degrees every decade but by the end of the 21st century it won't be as high as you would otherwise infer from my previous statement >> well one >> presumably he thinks because the technology will draw down CO2 levels or whatever I don't know but we're doing what we're doing and it's going up by 0.2 to every 10 years. That's what we >> He does work on carbon capture. So, I mean, he thinks that that is going to come through. >> Sorry, a little detour there. Very interesting. I like it. >> This is serious. This is serious, folks. It's going up by 0.2 degrees a decade. This is serious. This is not some crazy theory. It's happening. >> Yeah, but Bill Magcguire is at the extreme end of predictions. I think that is important to note. Just >> It is important to note, but bear in mind this guy >> doesn't mean he should be dismissed. It's not the central estimate of the majority. He's very credentialed and of course very credential people have been wrong about all kinds of things but he you know he he was a member of Sage under the black government he was a a professor of volcanologist and a climate science professor at UCL he's a serious guy you know he has personally moved to the peak district a thousand feet above sea level cuz he thinks this stuff is real and it's coming in the next 20 to 25 years in this country. Yeah, personally moving to I think that might suggest someone who tends towards the catastrophic, but so do many of these people in Silicon Valley that I take very seriously. So I I agree take this guy seriously. I'm just saying the central estimate just as the central estimate isn't that AI is going to kill us all by the way. Like that's I mean [laughter] the average seems to be in Silicon Valley they think that there's 10% chance that it's going to kill us all. That seems to be the central estimate. >> Well, the central estimate of AI and boy we're going over here. The central estimate of AI is that it's just going to make regional and income inequality crazy. You thought it was bad already. That's the central estimate. You know, the worst case it's going to be the matrix or Terminator, too. You know, middle case, yeah, you know, we'll we'll kind of have inequality repeat what we've seen over the last 25 years. And that gave us Trump and Farage and the AFD. So, God knows where we'll be by uh 2050.