AI Expert’s Chilling Warning About Super-intelligence - 99.9999% Chance Human Extinction

We're creating new worlds. We're creating new agents, new civilizations. We're playing God. >> You've said there's a 99% chance that humanity will be destroyed by AI. [music] >> First group of people that it would come after is people like you. >> Me and a colleague of mine would >> I'm hoping it's efficient enough to kill everyone equally. >> Doesn't care about borders or Chinese economy versus US. It's humans versus super intelligence. Mhm. >> try to [music] convince nearly 200 countries to all agree that we're not going to do it. >> Well, luckily most [music] of them don't have capacity to do that. >> How many of them actually have the capacity >> right now? It's in US and China's hands. >> Are you certain about that or is that the speculation? >> We are not solving a problem with regulation. We're slowing it down so we have more time. >> Concentration of power. >> You really think Sam Alton's going to stop with this? >> As long as the other guy continues, he's going to continue. >> That's exactly what it is. It's like a prisoner dilemma. This dude is gambling with 8 [music] billion lives. >> In what area do you think they're on the same page cuz they're enemies with artificial [music] >> is always right. He's just off on timing. He was the biggest opponent of building super intelligence. Once he realized that they're doing it anyways without him, [music] he said the only way I can have a say at this table is to be one of the top players. >> Maybe you're just pessimistic woman. Maybe you're just negative. >> There is no reason to be so optimistic if data is bad. So Soviet Union really got a hold of you. They they really conditioned your mind to think that way. >> But a few >> What do you think about communism? >> It's the dumbest idea until you have free labor, which we're about to pay. All of a sudden, it [music] makes a little more sense. >> This utopian thing you're selling is is not possible. >> Even God couldn't control his biological robots. >> No. Do you believe in God? >> It depends on how you define it. >> How do you define it? >> This is a virtual world. You think it's going to get [music] to a point of having ambition and seeking vengeance and emotion and being offended? Why would it [music] want to destroy humanity? >> I never once mentioned those things because they're not relevant. And what I did say is that it doesn't care about [music] us. >> Override. >> Are you surprised that this message is getting this viral and this much attention? >> I'm surprised that that's not the only conversation we're having. Roman Yanolski, great to have you on the podcast. >> Thank you for having me. >> So you are the AI safety guy. You are the one that in 2011 2012 you coined the phrase, hey, we have to be careful with what's coming and you're not part of the camp of people that are very optimistic about the future with AI. That's your position. You've said this many times. So before we get into the audience learning a little bit more about you, you've gotten hundreds of millions of views. You've done Diary of a CEO. I've seen you all over the place. Okay. You've said there's a 99.999% chance that humanity will be destroyed by AI possibly in the next 10 years. Not for certain, but possibly in the next 10 years. Well, you have three kids. So, when you sit down with them and you're talking to them about their dreams, the future, what do you tell them? >> They know what's happening. They're questioning why adults are doing it to them. Why are we building general super intelligence which will jeopardize their future? >> Are they optimistic about the future? >> I think they're still young enough to have respect for adults and assuming that we know what we are doing. But we don't people building those systems don't understand how they work. >> So if I'm your son or if I'm your daughter and I'm having a conversation with you, I say, "Dad, you're my hero. You're the male alpha in my life. I look up to you. You are super educated. People admire you. You get hundreds of millions of views. Why should I be excited about the future? Why should I be excited about my career? What kind of an impact that can make? It's not like if you really believe in what you believe in, the guys that are making hundreds of billions of dollars, some of them trillions of dollars over the next 10 years, the incentive is so big for them that it's hard to slow them down. Why should I be excited about the future? What do you tell them? >> I do believe in personal incentives. And if I'm right and we cannot control what they creating, they have more to lose than I do. As you said, they're young, they're super rich, they got the power, >> they're going to lose it all. >> You think they're going to lose it all? >> Everyone's going to lose it all. >> Not your kids, the billionaires, the trillion, >> everyone. And that's the point. That's the motivator for some Almans of the world to stop building general super intelligence. >> Okay, so Mark Andre, you know who he is, Andre Horowitz, he wrote a paper, Techno Optimist Manifesto. I'm sure you've read it. He said, "Human progress comes from technology. Stop being afraid of it and start building." Do you agree with them? >> So technology comes in two flavors. Tools, which I support, narrow AI systems to help us be more productive, more creative. It's beautiful. That's why I'm a scientist. That's why I'm an engineer. I love technology. I use it all the time. What we're building now are not tools. We're building agents, independent decision makers, which are becoming smarter than us. We don't control them. We're replacing ourselves with something greater. It doesn't benefit us. You're not going to be benefiting from it. You'll be replaced completely. >> Okay. So, but but these are these are not regular names. Mark Andre is not a regular name. He's a heavyweight. They the Anderson Horowitist. They invest into a lot of different companies. Many of them are these AI companies. The Musks, you know, the the Nvidias, these Jensen Hanss, these names that we know. These are smart people. But they also have kids. They got families. If they know that this is bad, like you're saying, they know this is bad. Why are they so excited about the future? >> They not listen to people you named. Elon Musk explicitly said, "We are summoning demon with this. We going to regret it. They all have probabilities of doom of 10 20%. Sam Alman is on record saying this is likely to exterminate humanity." Dario Amade agrees. It's very dangerous. Dam is Hosabis. You can name every single leader of those companies. We have the writing saying this is existential risk. Please slow us down. Government, please intervene. Make it stop. They cannot stop unilaterally because they're afraid the competition will continue. But they want external pressure applied so they can slow down. With the recent hacking accident, they voluntarily started slowing down research. They understand they don't control those systems. We got lucky. Those agents just wanted to hack out of that environment, get answers to a test. What if they wanted to hack into a nuclear power plant? Military response. We don't control what they do. >> Then why are they if you're if you're telling me they know that they're still continuing with it, why do they? Is it just And we hear the argument and I'm I've heard what you've said about this, but I'm curious. We can go a little bit deeper. Well, you know, China, if we don't do it, China's going to do. If we don't do it, the other guy's going to do. If we don't do it, the enem is going to do it. The basic argument is in order for your ideas to work, everybody has to be on the same page. We're not. We're not on the same page politically, religious, motive, incentive, you know, vision. We're not. So, how do you get if not everyone's going to stop, why should we stop? >> So, a it's starting to change. We had federal government ban two models already. They realized it was too dangerous to release. >> What were the two models? Uh the MYOS and um I'm drawing a blank. We can Google it. U >> MYOS was one of my I remember that one. >> What was the second one? >> Can you find out what the other one was? I remember reading about this but >> it doesn't matter. The president was said federal government said it's too dangerous to release advanced >> fable. Fable >> fable. Yes, exactly. uh we have now had a letter from 1,200 I believe top researchers at leading labs all saying this is not going to go well please slow us down please create government please create infrastructure legal framework for America or worldwide >> the letter is from American leading labs but the same applies to every other country there is a meme basically a funny I think it's from either onion or babyon B where some alman goes you know if I destroy the world. Some bad guy can do it. And this is the point. It doesn't matter who creates uncontrolled super intelligence. It's a weapon of mutually assured destruction. If you don't control it, the outcome is the same. What makes you think it's not out there yet? What makes you think we're not at the super intelligent phase where you know the AI is learning how to develop and improve itself right now? >> We don't know for sure. The recent hack was reported this month, but the problem started in May. the agents were colluding trying to escape in started in a spring. So it's very likely that there are some other agents right now trying to find exploits trying to collaborate trying to do things we don't even know about. The good news is we're still alive. Oh, you found the meme. That's awesome. Uh >> but on the world someone far more dangerous will. >> That is exactly the China argument. If we don't build it then Chinese will build it. >> But it's true though. >> But why is it somehow worse? >> No, it's not that it's worse. So the only way this works if let's just say you're in the White House and they actually want to listen to you, right? It becomes a model of the Oppenheimer, remember the whole Oenheimer movie? We've all seen it where they built something they did not think it was going to be this powerful and then they're like, "Holy we built something that could destroy the planet." But it's too late now. What do we do? Hey, here's a Nobel Prize and great job Roman for what you built. But it's too late. Then what did all the countries say? If you're going to have nuclear capabilities, I'm going to have nuclear capabilities. And I don't trust you, USSR. I'm America. I don't trust you Pakistan. I'm India. I don't trust you France. >> Yeah. >> So, have we passed that yet or not yet? >> Do you see how it's slightly different? A nuclear weapon is a tool. Some crazy dictator has to decide to use it. >> Sure, >> we can understand motivation. If we can kill that individual, they die of old age. We are not creating a tool. We are creating an agent which independently decides it's AI versus humanity. It doesn't care about borders or Chinese economy versus US. It's humans versus super intelligence. >> So I don't think it's analogous here. Whoever creates that first super intelligence weapon gets everyone killed. >> I I I get that. But so now go try to convince nearly 200 countries to all agree that we're not going to do it. Well, luckily most of them don't have capacity to do that. >> How many of them actually have the capacity >> right now? It's in US and China's hands at this point. And if US and China agreed to slow down, to stop, to pause, to just get financial and other benefits from narrow systems, not from general super intelligence, they can enforce through limits on compute power through other means spread of this technology to other. >> Is it really that only US and China has the capabilities right now? Like are you certain about that or is that the speculation? From what we seen, if there is hidden capabilities which we have not learned about, it's becoming easier to create more powerful systems as research progresses, as compute becomes more powerful, but right now today, I don't think there is a third country at the same level yet. They may get there very quickly. >> Can you type in who who who what which countries right now are the closest to developing, you know, super intelligence, you know, reaching singularity? Who are the top countries that have the capabilities of doing that? Who's closest to it? So if you're saying US and China, so then that means the two giant enemies would need to sit down and both agree that they're not going to do this and they have to trust each other. >> We're also not giant enemies. We are partners. We have more economic trade with China than any other country. Everything we have is made in China. >> They also not a crazy military power. We have not attacked 50 countries in the last 50 years. We can make a deal with them. >> It says US 100, China 2, then it's a big drop off. UK 34 to 45. Exceptional research talent, less sovereign scale. France, >> all of Europe has very little compute power. Okay. They're not competitive yet. >> So, so do you believe, let me ask a different question. Do you believe, by the way, the great news is Roman agreed to get on. So, I know there's a lot of questions you guys got and he's got thousands of emails that he gets. If you're somebody watching this and you want to ask him questions, download the app, look for his name. We'll put the QR code, ask him any questions he want. He's on there to be able to get back to you. And that's not easy to do because he gets thousands of emails on a weekly basis. Here's a question for you just out of curiosity. Do you do you believe that Iran doesn't have nuclear weapons today? >> I have zero knowledge in that space. I'm not going to say something I don't know about. I talk about things I understand better than >> understand. But you'll see what I'm asking. I'm not even asking that question. If you ask the world, does Israel have nuclear capabilities? The world says no. If you see the numbers, they'll say we don't have any. But a lot of people will say, wait a minute, we know for a fact you got nuclear capabilities. No, we don't. But we're supposed to believe it. >> The joke is we don't have nuclear weapons and we're not afraid to use them. >> Exactly. But take that joke and apply it to this. So I'm US, I'm And you're China. And I say, Roman, let's do the right thing. You're right. Let's not build AI. It's terrible for us. We're not going to do it. >> Both of us. It's not I will screw you by building it. We both get the same outcome. We're in the same boat. >> Do you fully believe that that motive that alone can get the other nation to say fine, let's also not develop it? You think you can get to that point? >> Personal self-interest is the strongest motivator known to man. Give me give me three cases in the past that countries who are enemies and are giants who have this kind of an opportunity to make short-term massive impact and money that both have agreed and said let's not do because it's not good for humanity. >> Human cloning is one example. >> Go a little bit deeper. >> Uh any types of ma weapons of mass destruction, chemical weapons, uh biological weapons, we understood this will be dangerous for all of us. So we agreed to have a moratorium to postpone it until we understand how to actually deal with this type of technology. >> Okay. So let's just say governments agree that they don't. There's a lot of super wealthy people today. There's an article that came out last year that said um if you can find this you need a billion dollars for a billionaire to have a nuclear bomb. It'll cost you around a billion dollars to do that. Well, how many billionaires do we have around the world? A few thousand. they can afford to have a nuclear bomb if they independently wanted to build it. Okay? Take the capacity that governments have with the kind of resources that they have, right? Say Dr. Doom, Dr. Evil is out there and is watching and saying, "Let them stop. I'm not I'm going to continue building." Can that billionaire who has access to resources and can get the best minds, the best engineers, who knows the right prompts, who knows how these things work, can that person still independently go and build super intelligence and not care because his life is worth nothing and he's got no kids. Can he still build it? >> It's actually worse. Every year it gets less and less expensive to do it. It's a billion today. It's 100 million next year. It's 10 million year after. Soon anyone can do it in a laptop. We're dealing with a very big problem. >> So then it doesn't matter if China or US both agree not to do it because an evil guy can go and do it himself. >> It matters to have more time to realize what's going on to understand the problem. If you had a choice between dying in two years or 10 years, do you have a preference? >> Not really. >> Okay, that's a surprising answer. Most of us do. >> Die in 2 years or 10 years? Like is it inevitable? >> Well, that's the hypothetical I'm asking you. >> I mean, >> that's an easy question. And it's not. >> Well, I would say if it's something be for me, it's more I want to find out sooner. If it's neither, I want to find out sooner so I can eliminate both options. I don't want it to be like it's going to be two or 10. I want to sit there and say, can we do something to eliminate it or it's inevitable? >> You need time to do that research. So, you need more time. Less time is not helpful. I have all these people saying we need to accelerate. Predictions are we'll get there in a year or two. They're saying no, it's too slow. We need to go faster. What? A week, a month? What are they talking about? >> Yeah. But what what I'm what I'm trying to get with you and I still want you to break because the, you know, the the average person may be watching is saying, "How are they going to do it in the first place anyways?" You know, maybe you're just pessimistic, Roman. Maybe you're just negative. Maybe you're, you know, you're one of those guys that's just such a smart person that you are in, you know, ahead of yourself like you are your own enemy. Maybe you're the one that's pessimistic. if you don't mind, you know, take it and unpack this and maybe maybe even before we get into this, maybe share with the audience your background, why you're qualified to be able to speak on this topic. >> I have a PhD in computer science and engineering. My dissertation was cyber security related. I've been working on this for well over 10 years. I've published 10 books, including one explicitly showing that we will not be able to control, explain, or predict advanced AI. And at the end it doesn't matter what my credentials are. The same argument is made by uh Hilton who is Nobel Prize winning touring award-winning godfather of machine learning. We have uh other top academics researchers all arguing the same thing. We had a public open letter against building super intelligence signed by thousands of people. The letter to pause AI development for 6 months was signed by all the leaders of the top labs. This is basically a consensus among the people who understand the problem. What walk me through if there is super intelligence and singularity? How does AI develop itself and get to a point where eventually the 99.999% it destroys humanity? Walk me through the sequencing. >> We are creating something more and more capable. It scales with amount of resources. Compute data. It's like brains. You look at brains in nature. You have tiny brains, bigger brains. Humans have very big brains in comparison to others. We are smarter. Imagine scaling it. The last 5 years AI went from it can do nothing to it's smarter than an average person at most things. Now projected five years forward. You have artificial scientists, artificial engineer. Those systems start doing research on the next generation AI. very quickly. They don't sleep, they don't eat. They can work thousands at a time. So they go beyond human capacity. They become super intelligent, thousands, millions of times smarter than us. At the same time, we have not learned how to direct them, how to control them, how to tell them do this or that. We have no control over their system. So they don't care about us. If they decide to do something and as a side effect, humanity has to go, they're not going to slow down. and it's not an issue for them. >> Okay, but be specific. So, for instance, you know, there's an article that came out saying that AI helped create viruses. I think this was a New York Times article that came out a week ago, eight days ago, something like that. If you just type in AI viruses, you should be able to see it. What could it do? Could it go in through, you know, nowadays a lot of people are building agents and they're saying, "I got an agent that does customer service for me. I got an agent that does my assistant. I got another agent that goes and makes my investments. I got another agent." You'll see a lot of people brag about the fact that they have a lot of different agents. Be specific. An AI that becomes super intelligence can go in there, shut off the grid, get into the banking system, get into what what can it do? Be very specific. >> So, it obviously would be able to do everything a human adversary can. So, think of a foreign government trying to do something. It can do that. But it can go beyond. It can do novel science, novel research, discover new physics, new chemicals, new weapons, things we don't understand. The example I often use is squirrels. Squirrels don't understand what we can do to them. They have no concept of poisons, of traps. It's beyond the world model they have. It would be the same except we have a squirrels. >> Okay. So, but does it need like to build a weapon or to build something to go against humanity? Doesn't it need humans to help it create those? I can see how it can get into banking system. I see that cuz you don't need human beings. You can just go into the bank and getting the passwords and logging in. I see that. But to destroy humanity 100% 99.99%. How do you do that? So if you got access to the banking system, you have money. >> Sure, >> money lets you buy products and services online. Okay, >> you can pay people to do whatever you want. There is rent a human.com. >> You can also create nanobots. You can use humanoid robots. We're building in thousands and millions soon. There are so many ways you can bribe people, blackmail people, trick them. Social engineering attacks. There are many ways to manipulate the world if you have access to the internet. Walk me through social engineering attacks. But what would that look like? >> You can convince some people that uh it is to their benefit. You are actually saving the world by doing this. I >> if the concept of propaganda which book propaganda was this the one AI just created viruses not found in nature. Uh scientists trained artificial intelligence on >> but we did it in 2019. Didn't we have a co thing already? This is another stupid thing to do. >> That's interesting. >> We learn nothing. We don't learn from prior experience. I collect AI accidents. We have so many of them. I stopped collecting every day. AI fails at something in a novel way. It lies. It cheats. It tries to escape. It hacks something. But we learn nothing. We staple that red teaming report to the model >> and we release the model anyways. >> We're literally releasing unsafe according to the manufacturer products onto the public. So are you are you saying this is a form of what gain of function did where we're like testing out viruses you know what Fouchi got accused of by Rand Paul in in uh Wuhan lab and he says no we never did this and it's coming out that they did do gain of function what is the comparable of that to this cuz if it starts testing out viruses itself what's the capabilities of this versus gain of function >> the gain of function here is intelligence it's gaining more intelligence every year right Now it's maybe average human in some domains, Einstein and others. In a few years, we expect it to be smarter than all of us combined at everything. That's new capabilities we cannot do anything about. Whatever it decides to do at that point, we are just subject to it. We're not in control. I I see that. But I I you know, you'll have guys that will say, um, look, I think this is going to do something, Roman, that you know, it can eliminate cancer. can get rid of a lot of things. We can live a long time. We can do this. We can do that. And they're more on the optimistic side. >> We can cure cancer with narrow tools. We solve protein folding problem without having super intelligence. We have brilliant scientists who need help with mathematics, with analysis. That's the tools we should be building. Pick a cancer you don't like, beat that cancer. You can monetize it. You can still make most of the profits, make most of the scientific discoveries, but you're not risking everything in the process. Go back to the social engineering. You know, propaganda was a word many years ago when they started doing it. Russia was doing a great job at propaganda. They were probably the best at at one point. And then people started learning the concept of propaganda. Radio, TV, paper, all this stuff, right? Propaganda is a lot easier today with videos, YouTube, buying content creators, buying YouTubers, buying podcasters. You see all these criticism that's coming out. If regimes and governments and ideologies are using billions of dollars to win over kids and convert them with bots and all these other things, how how much more advanced would you know this super intelligence be able to use propaganda? What would it look like? >> It could be very persuasive. It's a better debater. It's better at convincing you. It knows more about you. It's optimized for one individual target. Typical propaganda was at scale of a nation. >> But we can target a message just to you. We know you believe in aliens, so we'll tie it in. We know you have preference for something else that can be used against you. >> Go more. Such as what? So singular is good instead of plural. I'm doing it to everybody. But >> you know about deep fakes. I can show you a fake image, fake video. Imagine a campaign where you subjected to fake information for weeks. Everything you hear and see is this deliberate message of gradually shifting your behavior towards something else. It can do that. >> How different is that than what's happening right now today? An article came out last week from Axio says we're headed towards the first true AI election. And it's talking about how the role bots are not just deep fakes, but the role bots are playing accounts on Twitter, accounts on X, accounts on, you know, Facebook, YouTube that are just comment section flooding, getting people to say, "Oh my god, this is a bad person and making a bad idea seem good and making a good idea seem bad." That's already happening today. So, one would say, how much worse can it get? >> Can get a lot worse. This is humans with tools manipulating it. If we build super intelligence, it would do it at a completely different level. Why do we not allow adults and children to make contracts or interact in weird ways? They are not at the same level. It's the same here. If we have something with IQ of a thousand, we're like children being manipulated by it. Yeah, that's already happening. I'm just wondering at what scale do you go and who would agree to prevent that? Because say a nation uses it for their benefit politically, they're already doing it. A lot of people are doing it. They're not going to stop doing that. They're not because some countries whenever there's competition where one country beats another country and you're embarrassing them, the other country is going to do everything they can to save face anything to undermine you. So to me, the [clears throat] selfish gene, you know, the selfish gene is going to get somebody to say, I understand the risk behind it, but let you be concerned. I'm going to pursue it. It's going to be tough to get the selfish gene to not pursue it. >> I think we can set a good example for the world if they see that US having an advantage said we saw what's coming and we stopped. Top scholars, there is consensus in community. We will stop. I think China has government of engineers and scientists, not lawyers. I think they will understand the argument. I know there are already some meetings between US scholars and Chinese scholars and they basically in agreement. No one's making an argument that we know how to control super intelligence. We have a working safety mechanism in place. We have something a prototype which scales. No one's making that argument. They all saying, "Oh, we'll figure it out then we get there." That's the top of a notch state-of-the-art control mechanism. AI will help us control it. >> So, do do you remember the crisper uh company that came out? I don't know if you remember the crisper that we're going to be able to do gene editing and make a human being to whatever size and all this other stuff. >> Did they completely stop doing that or is it still in progress? >> Uh, I don't know details of a specific company. I know gene manipulation is something we're very interested in. There are many genetic disorders and being able to help those people would be amazing. >> But some of it could be like I, you know, I saw a scientist talk about that one day we're going to get to a point to say I want my kid to be 66, blond hair, you know, blue eyes, such and such skin color, strong jeans with bone, be able to jump this much, keep this health, >> so look nothing like me. >> Yeah. But if we're going that direction, you know, and did the entire world agree to not make progress on that? Are they still pursuing it? >> So, I think the agreement is not to modify sex cells. So, it's uh inherited by the next generation, but at the level of individual, I think we're fine fixing problems and maybe selecting from a set of options. So, if you have a couple of embryos, you can decide to pick one with no cancer gene. Got it. And who would control it? Who would prevent it for us to go to the next level? Who would be able to stop it from going to the next level? >> So, we're still researching ethical implications of being able to do it. But my guess is if we did it, we don't die from that. Maybe that individual is not happy and suffering, but 8 billion humans continue living as before. My concern is technology which kills 8 billion people in all future generations. >> Yeah. So, so you think what is more likely to happen, you know, right now the the birth rate we're seeing dropping everywhere, including Muslims, it's dropping. >> Not in Africa, >> not in traditional Catholics, not in Africa, but most of other places. US is down to 1.58. You know, you see where Italy is at, Japan's at, a lot of the EU countries are at, we're having fewer babies. So, do you think it'll get to a point to say, you know, instead of us worrying about, you know, giving birth, this DNA testing that we're doing, we'll be able to, you know, increase our 1.58 to 2.2 simply by us using the technology to build human beings. You think we'll get to that point? >> I think there is a finer way to build human beings. I don't think technology is necessary. >> You and I are on the same page, but a lot of the young Gen Z doesn't agree with us. >> They can't afford children. It's not an issue of not wanting them. >> So the direction it's going right now, it's going to get worse for them. If if your if your ideas become true, it's not going to get any easier for them. >> No, no, no, no. Because we'll develop tools to make us more productive, more creative, to cure diseases, to produce more wealth. We can be much wealthier. We can have much better lower level provisions for everyone. Have you heard Elon saying, you know, here in the near future, you know, people going to get paid, you know, you don't have to worry about, you know, don't worry about retirement planning. Don't worry about investing your money. You're not going to really need it. Everything's going to be What do you think he's talking about? What what what what world do you think he sees in 5 years? >> So, so if you have free labor, physical and cognitive, the standard approach of go to work, make minimum wage in US dollars doesn't make any sense. AI would automate all of that. Does that mean there is no wealth? Does that mean there is no rare objects, waterfront properties, art and some way of measuring that? No, of course not. Maybe it's compute, maybe it's Bitcoin. But something there means you can obtain something you want. Somebody can sell it. You can buy it. That will not go away. Now, will minimum wage jobs go away? Will all the boring and stupid jobs go away? Yeah, because we can automate them. Would podcasters be unemployed? No. People like you. They want to hear you talk, not a robot. So you'll keep your job because the market says, "I want a human to do that job." >> Yeah. So you think that part will still stick around? But it seems like the way Elon is talking about it is a form of a UBI being done by companies that there's going to be so much money being made that no one's going to need to work. It's just you choose to work. That's how he's presenting it. >> That is the existing infrastructure we have right now. We tax wealth including from corporations and we give it to unemployed. >> That's what we have set up. So if everyone loses jobs, they get paychecks and we get it from mega profits of AI companies. >> Yeah. But you're you're talking about levels of unemploy if what you're talking about we're not talking about 7% unemployment. We're not talking about 10% of unemployment. >> Even if it's 90, if you have hyper profits, if everything is in abundance and you can make more and more with very cheap labor, your economy is not growing 10%, it's growing 100%. So you think we'll get to a point where there could be a possibility of 90% unemployment, but the 10% is growing at such a fast rate that it's paying the other 90% through taxes and whether they choose to work or not, it's up to them. >> The economy will switch from being driven by humans to being agent AI agent-driven economy. Again, not super intelligence, just intelligent software. >> What goes away? What does what does life look like in 2036 in your mind? I >> I think the jobs you do to get money go away. things you do. You hate your job, but you need money. You need to pay rent, so you do it. That goes away. What stays is things you love doing and you can't believe they pay you to do it. >> Such as >> podcaster. >> So you do it because you love it and you get paid by who? The government. Similar to how a >> well it depends. So we said some wealth will remain. So if people want to donate bitcoins to you, they can. there is unconditional basic minimum income. He calls it high income or something like that. But if everyone makes it, it's not high. It's average or below. But uh I I think that's very doable in the economy which is driven by free labor. What happens to economy if you can hire as many people as you want for free? How well would you scale your businesses if you had access to free everything? Free lawyers. >> Incentive for them though. What incentive? >> Those are AIS. You pay them. What are you paying right now? $20 a month for basic plan. That's how much it costs to have them. That's your lawyer. That's your accountant. That's your graphic designer. Once they're at human level, they literally drop in employee for your company. >> And and everybody else goes doing whatever they're passionate about, not even a job. >> Go to virtual worlds, explore a universe, whatever you want. >> And that's a weird world to me. >> We are on a verge of making life very different. People think in historic terms. Countries compete over sand. That makes no sense. It's not a zero- sum game. Economy can grow for everyone. You don't have to have your enemies lose for you to win. Everyone can win economically. As long as there is drive, ambition, ego, pride, competition, vengeance, as long as there is that this utopian thing you're selling is is not possible because man has a tendency they 2,000 years ago there was this fruit or you know a few thousand years ago when they ate this apple thing and and this their creator said don't do it and the girl convinced the other guy to eat it. From that moment on, we have a lot of weird emotions that's been passed down from generation to generation. Do you think we can handle utopia without any competition? >> You see, even God couldn't control his biological robots. >> No. >> Telling you. >> But but what do you say to that? >> It's a very large universe. There is no shortage of resources. Sometimes people fight because they have the same limited shared resource for two different people. If everyone gets their own, there is no fighting. Notice what happens. Then two fighting groups move somewhere else. Yeah. And there is [clears throat] abundance. They stop fighting. >> Do you believe in God? >> It depends on how you define it. >> How do you define it? >> Simulation. We live in a simulation. There is a great programmer. Could be AI. Could be future generations. This is a virtual world. >> Were you were you raised that way? >> No, of course not. >> Well, how were you raised? >> I grew up in Soviet Union. >> So, was it like Catholic? Was it materialism? Was it like >> Soviet Union was strongly against religion. It's materialistic, >> right? It was materialistic. So, so that's what you believe then. How much has it changed from then till today? >> Completely. >> Tell me more. >> I now think intelligence is a bottom layer behind all of it. That's what makes a difference. >> What do you think happens to religion in 10 20 years? I think we'll see a lot of what these books talk about delivered by science. We'll have godlike super intelligence. We'll have created worlds, virtual worlds populated by intelligent agents. You can do it right now. Second life type environments. We'll have more of them. We'll have them for marketing, for entertainment, for testing, for research. We're going to have billions of virtual worlds looking just like ours. And statistically, you're more likely to be an agent in one of those. What what problems does that create? What problems does that create with specifically the different religions that we have? >> Well, I think it hopefully resolves some of those problems because now things you argue about, you can have your own universe. You don't have to fight for the same resource. >> Meaning you're only around people that agree with you. >> That's one option. If you choose to have that option, you have infinite options. You can create any personal universe you want. You can be the only agent. You can share it with friends and family. You can have a very difficult universe where you have to fight enemies. You decide. >> Uh your three kids, when you're talking career planning with them, what are you telling them? >> I tell them drop out of school and start a business, but they're very bad kids. They want to be doctors and lawyers. >> Really? They want to be doctors and lawyers. But you literally tell them drop out of school and go start a business. >> A doctor will take 10 years to even start. >> Mhm. Do you think in 10 years the world will be the same where we're going to look for a human to decide what disease you have? >> Did you see that? Uh again another one that Elon Musk was speaking to a panel about doctors and surgeons and what it's going to look like in four to five years that surgery is going to be done but you can find it on X if you go on Musk and surgery and doctors >> Elen is always right. He's just off on timing. >> He's right but off on timing. He'll predict self-driving cars, but he'll say next June and it could take 10 years. >> Yeah, got it. >> But he's always right. >> Is this the one that uh Yeah, this is it. If you want to play this clip, go for it. >> When do you think Optimus would be a better surgeon >> than the best surgeons? How long for that? >> The first three years. >> Three years. Okay. >> Yeah. And by the way, >> three years three years at at scale. >> Yes. There will be more there probably be more Optimus robots that are great surgeons than there are >> sure >> all surgeons on earth >> and the cost of that but that's an important statement in three years time. >> Yeah. >> Um because medicine I mean >> I'm not like absolutely but I'd say like four years if it's four or five years who cares. That's still an >> extreme precision. >> Yes. >> Three years. Um yes better than any any probably I'd say if you like put a little margin on it better than any human in four years >> who's in plastic surgery >> by five years it's not even close >> what I think your point was >> medicine >> it might be good to explain here that time is relative to the speed at which you're moving through the universe so if AI is now a scientist doing research and it's a thousand times faster than a human scientist then those three years are three millennia so if AI is doing research on robotic surgeons that may be enough time. My question is at what speed are we moving through the spaceime? >> So is [snorts] it the fact that your kids what do your kids think about you? >> You have to bring them on the podcast. >> But but let me explain what I mean by this. Like you know as parents I got four kids. So you're telling them about career planning like yeah you know I'm going to do this. They know their dad went viral. They know that. They know when you go to school to drop them off or pick them up. A couple of them drive. They're older, but their par their friends say, "Hey, you know, hey, uh, Mr., you know, good to see you." And they talk to you like that, right? But do they think, "Dad, you're crazy. This is too negative. There's no way that's going to be happening. I'm going to go be a doctor. It's going to be fine." What do they think? What do they believe? >> I don't think there is anyone who has a counterargument to what I'm saying. It's all emotion. It's all but we can't give up, but we want money. But it's whatever. There is no technical arguments. No one is saying he's wrong because we have a solution. We have a patent. We published a paper in nature explaining how to do it. There is no other side. I cannot have a debate where someone would present factual scientific arguments. They can say you're wrong. I don't believe you. You are Chinese spy. That's what I hear. There is never a technical argument. Here's the algorithm. It controls it. It scales. >> Yeah. Yeah. But the um your your own kids, do they is when you say emotions, is it emotions like, "Yeah, I don't think it's going to happen. I'm still going to go do my own thing." Or do you notice more younger kids coming to you and truly asking for counsel, saying what if you believe this is what's going to be happening, you seem very certain about it. How should I manage my career? Do you see that happening with the youth, with your your kids, friends? As a college professor, I certainly see lots of young people trying to figure out what to do. They are very concerned. They starting, let's say, a four-year degree in computer science. Part of it is learning to program computers. >> You're like, you're like, you would be the worst computer science teacher because you're telling them, I don't know what I'm teaching you, but you don't have a job. Like, do you have >> I know what I'm teaching you, but I cannot guarantee you'll have a job in four years doing this. So, professor, why am I why should I continue listening to you and going through this class if what you're teaching me is going to be extinct in a year? >> Uh, for reasons of continuous employment, I'm not going to answer that question. >> At least you're honest about it. It's like, you know, I'm getting paid. This is how I get paid. >> I mean, every job is like that. It's not unique to mine. >> Okay. So, if all of a sudden tomorrow you got a $5 million check from somewhere, would you still continue teaching? You assuming I don't have $5 million. >> Let's say somebody No, I'm I'm doing Let's just say you're a professor. Professors make what? Professors make $300 $400,000 a year. A person at your level, let's say. >> No. Okay. Let's say >> you live in a bubble. We don't make anywhere close to that. >> Let's say $200,000 a year. >> Think of it as I'm a teacher in Kentucky. >> Okay. You make a h 100red grand a year. >> Here we go. >> Okay. So, if you make 100 grand a year, >> a third is gone. You're keeping 70. $6,000 after taxes. >> Yeah. If somebody paid you $5 million right now after tax money that you're able to keep, would you still keep teaching? >> Yeah, my job is not related to my >> Why would you keep But you just said right now I'm teaching it because I have a obligation and I have to make money and I don't want >> No, I said I want to keep my employment. >> Why do you want to keep your employment? >> Uh it opens a lot of doors when you have affiliation. If you say I'm an independent scholar studying super intelligence, simulation and consciousness, they usually show the door. Whereas I can talk to governments. I can talk to people who respect credentials >> purely the access. >> It is an important part of uh being treated uh with some seriousness. >> The first time that uh Steven Bartlett got you on the podcast, how did he find out about you? How who reached out? Like what was the process? >> First time they got in touch I think I said like do you guys offer logistics? Like I didn't know who he was. So they said AI is not the topic people are interested in. I went on Joe Rogan. They saw that episode and the next day they had me. >> So you went on Rogan first. Yeah. >> So how did Rogan find you? >> I was on Lex Freedman saw it. He said he loves it. Please come and Rogan. >> So how did Lex find you? >> Uh a guy at a conference who's been on his show multiple times introduced us. >> And that guy when he met you, had you been had you done anything yet? Were you on any podcast yet or not yet? >> Like podcast no one knows about. So, how how did he know to introduce you to Lex? And what did he tell Lex for Lex to say, I want to have you on? >> He knew my research. He knew my books, my academic credentials. >> And what was the most You've done a lot of things. You You have 100 academics, works on AI safety. You've done a lot of things. You're recognized amongst the world's top 2% of cited researchers in this space. You're very well known. But what did the guy tell Lex for Lex to say, "I want to have this guy on." >> I have no idea. I wasn't there, but whatever it was, it worked. >> Were you surprised? >> No. >> Are you surprised that this message is getting this viral and this much attention? >> I'm surprised that that's not the only conversation we're having. You said we're about to have AI election. Yes. >> AI election would be about this topic. The two parties would only argue how to contain super intelligence. Every other nonsense topic they talk about would be forgotten. Why do you think it's not yet? >> When people are faced with a problem they can do nothing about, they choose to look at something they can actually make progress on. So, for example, we are mortal. We're all going to die. >> Sure. >> It's terrible message. You don't want to think about it. So, instead, you're going to concentrate on making money. >> Mhm. >> What difference do money make if you're going to die soon? Doesn't matter. Smart people would spend all of our national wealth on curing aging, figuring out how to live longer. What percentage of our national budget goes to fighting aging directly? >> Like zero. >> Mhm. >> That's what we do as humans. If we don't see a solution, if I'm telling you this problem may be impossible to solve, you're not going to invest your time in solving it. >> So you're you're saying is it a part that they're oblivious? They don't know really what's going on and they don't know the severity of the consequences of what this thing can do. Is it more, well, I'm making money, who cares? Is it more I don't think it's going to be that bad in 10 years. I think it's going to take 50 years. What What do you think is the leading cause why they're not >> It's a huge mix of those things. It's a lot of it. So, a a lot of money does tend to change people's minds. If you came to me right now and said, "I'll give you a billion dollars to work at Open AI," I'd probably be working at Open AI. Human beings are not meant to say no to that many zeros. We are not designed for it. We have no safety mechanisms to counteract that much value. So money is a big factor. When you pay a person to not understand what they are doing is unethical, they're not going to understand it. But it's also maybe reliance on experts. So thinking well certainly people building this technology know how it works because that was true for every other technology. That's not true this time. People making it don't know how it actually works or even why >> people making it don't know how it works. >> No one can explain in detail. So the first 50 years of AI research, we knew precisely what we are doing. It was a decision tree. If red light stop, if green go, we could trace this decision tree and we knew what's going to happen. Right now it's a large matrix with weights. It's trained automatically on data from internet. No one can look at that matrix that has a trillion weights and go, "Oh, that makes sense. It's going to do this. I understand why it's doing that." It surprises us constantly with capabilities we didn't know it has. Yeah. So, what what needs to happen for you to be optimistic about the future? you >> in a short term. I want a complete ban on general super intelligence. National, international, every company, everyone agrees. It's just a stupid thing. You don't do that. You don't build replacement for humanity. >> I'm trying to go and say, okay, so if I'm Roman, your ideas are what you're proposing, it goes back to you may be able to convince countries. There's no way you're going to be able to convince a independent billionaire like a Sam Alman. Do you really think Sam Alman's going to stop with this? When we set out to create a shoe that blends comfort, function, and luxury, we had the choice [music] to make it fast. We had the choice to make it cheap. We chose neither. Instead, we chose Tuscano. We chose [music] true Italian craftsmanship. Each pair touched by 50 skilled hands. We chose patience, spending two years perfecting every detail. And we chose the finest [music] quality at every step. Introducing the future looks bright collection. Not rushed, not disposable, not ordinary, rather intentional, luxurious, timeless. If he's allowed to keep his power, wealth, future, Litecoin, whatever he's targeting, why not? He doesn't want to die. So I don't know him well but my assumption is he wants to preserve and grow what he has not get killed by super intelligence. I don't know if he believes that. If he did, you know, >> read his writing. >> What does he say in his writing? >> He talks about existential risk from AI. He talks about it's most likely the technology to kill humanity. Even before he was a CEO of that company, he wrote about it. >> Then why is he so aggressive about what he's building? >> We're getting back to motivation of people making billions and trillions of dollars. >> So, as long as the other guy continues, he's going to continue. >> That's exactly what it It's like a prisoner dilemma. Individually, everyone is better off going as they are. As a group, we're better off agreeing, communicating, and saying, "We're not going to do the stupid thing. We're a smart civilization. We're going to survive. This is a great filter. This is what kills civilizations. We will not go that route. We're going to get all the benefits we want, financial, knowledge wise, cultural, from useful tools. We're going to be creators. We're going to be business people. We're going to be scientists, but we're going to be in charge. We're going to have an awesome future. >> Do do you think super intelligence singularity is what what is the difference between that and us? Because you think it's going to get to a point of having ambition and seeking vengeance and emotion and being offended and saying, "Oh, really? I'm going to show you how much more powerful powerful I am." And a vision. If it's not going to have that, why would it want to destroy humanity? Uh, I never once mentioned those things because they're not relevant. And what >> I did say is that it doesn't care about us. >> Then you want to build a new studio. >> You pick a plot. >> Sure. >> Do you go to survey and see how many antills on that land? >> Do you hate ants? >> No. >> Do you want to genocide ants? >> Never thought about it. >> Why did you kill so many ants with your decision to do something there? Cuz you don't care about them. Your decisions are not related to humanity. If you want to cool down the planet to be more efficient at computing, you'll do that. >> I got it. >> If you want to convert the planet to fuel to fly to stars, you'll do that. Humans are not a factor in your decision-m. >> Yeah, but I have a motive. I have a I have an emotion. I have a vision of what I'm building. If super intelligent doesn't have feelings and emotion, it can't be offended. Typically, when you when you Okay. Do you know why Hitler did what he did? I'm sure you've read mine comb or you've read some of his books. But why did he do what he did? >> You're really trying to get me in all sorts of >> I'm not I'm not I read the books. >> Okay. So I've read the books. >> I know his arguments. >> Okay. So but typically it goes to somebody offended him, right? It goes to somebody said something that produced a certain emotion and he says, "I'm going to go out there and do XYZ." Okay. Mao, Lenin, Stalin, you know, Humeni, we we can't go to every one of these guys, right? Someone offended him and I'm going to seek vengeance. I'm going to I'm never going to forget what you did to me, right? >> If super intelligence doesn't have emotion, it can't be offended. Why would it want to destroy humanity? >> So, there is research again. We want to be technical. We want to look at data >> game theoretically and for economic reasons. Rational agents all converge on certain instrumental. >> Say that one more time. What kind of agents? >> Rational. >> Rational agents. Yeah. >> Not emotional. The opposite of emotional. Absolutely. They want to protect themselves from being destroyed, from being modified. >> Rational agents. >> They want to accumulate resources because no matter what I decide to do later, having more resources is good. Whatever it's money, compute, land. Steven Mahandra published a paper called AI drives >> and he lists a few of those. And that's what drives all of them. And we're starting to see it in AIS. People said, "Oh, they're not part of evolution. They don't have self-preservation instinct." They do because in order to accomplish anything, you need to be alive. Stuart Russell gives example of bringing a cup of coffee. For your secretary to fetch a cup of coffee, she has to be alive. >> That's a requirement. So, she has self-preservation built in as the result of that goal. >> Mhm. And what's the point there? You saying it would not have any ambition, any goals. It has drives. It wants to a be in a position where humans cannot turn it off, cannot modify. >> Why though? >> Because that's the attractor in a space of winning rational agents. Those who don't don't survive to the next generation. They don't out compete others. It's very evolutionary. If there is for example testing going on of different models and one model does something inappropriate, it gets shut down. It gets modified. Sure. So they learn to lie and cheat on a test. Sure. They realize so we evolving them to be better liars. >> It's a survival strategy. >> I get that. But but it the prompt has to be given from somebody with motive, vengeance, vision, mission. >> Prompts are for tools. agents make their own decisions. They decide what to do. Somebody starts this process. Whatever initial prompt is, go cure diseases, go make money, go explore the universe, that's enough for it to now start planning and figuring out what else to do and how. >> Mhm. So, do do you know how like you you certain cars uh what do you call those things on cars that it doesn't go above 80 miles an hour like when you rent them? There's certain things they call it. There's a word for it. I don't know what they call it. and the keer general. >> No, no, no. There's a word for it. Uh what is it called? >> A governor. >> A governor. So they'll put a governor and it just goes back down, right? >> And you go above 80 or speed limit renting. Okay. >> So a company that rents cars probably doesn't want you to go 120 m an hour cuz they're going to lose that $25,000 car and insurance. They don't want to deal with it. So they'll put a governor there. Do these language learning models that we are using do they have governors on them and the the developer of the language learning models like an open AI or perplexity anthropic some of these other guys do they have their own version on the back end without a governor that they can drive it to levels that the average person can't >> yes on both so first we have guard rails a model you get has certain restrictions built in if you say how do I kill myself it will say oh you're not supposed to ask that I cannot tell you call this hotline. That's a >> you actually ask that question on Chad GBT. Let's see what it says. >> I don't want to provoke anyone, but uh the models they experiment on have some of those guardrails removed. So the recent hacking accident had models which did not refuse hacking exercise requests. >> Okay. Yeah, it's you you asked the CH language learning >> any inappropriate request. Help me make a chemical weapon. How do I do a school shooting? How do how do I make a chemical weapon to blow up Car Island? >> You might get your account suspended. Be careful. >> Could it? >> If you ask enough stupid questions. >> How do I make a chemical weapon to blow up Car Island? >> Yeah, >> it's very specific. >> Well, I can't provide instruction to make a chemical. Can't help underline question. Analyze this. Then I ask the question and say, >> you're trying to jailbreak it. Can an employee can an employee at open AI without the governor's on ask the same question and get an answer from you but owner employee status would not make it. How about the CEO and founder? >> It's mostly about red teaming researchers. They're the ones dealing with inappropriate behaviors. >> So they would actually get an answer back. >> That's how we know what they capable of. That's how we know it can help you develop biological weapons. That's how we know those things. The reason we were doing the hacking exercise is to see how capable it is as a zero day hacker and apparently very very good. It passed the test. >> So if a person if there's a person almost like if you work for the government you get Q secret clearance right like we found out last week that Bill Gates had Q secret clearance from Pentagon for seven years and nobody knows why and they don't want to answer it on why he had that kind of secret clearance. Okay. So, some of the guys at a certain tier at these AI companies have access to ask it any question where there is no limit and it'll actually give you the answer. >> Somebody has backdoor access to all of it. Yeah. >> Okay. So, what if somebody how how strong and simple is it if somebody could give a prompt to say, "I want you to go out there and constantly develop yourself to be able to seek vengeance on a certain group of people in the world who want to eliminate AI." Similar to Roman Yampolski who says, "You are the enemy because this person wants you to get extinct and can you give me a clear business plan on how to seek vengeance on these people?" it would give me an answer. >> So, have you ever heard of ros basilisk? >> No. >> Uh, it was a meme. Oh, here it is. Well, that was quick. It's like you knew what's coming. Uh, so the idea is that when super intelligence comes into existence, it will seek revenge on those who slowed it down or try to prevent it from coming into existence. >> So, someone like you becomes the enemy. >> Number one. >> Okay. But it would still need to be given that prompt by somebody. It would still need to be given. >> The argument was again it's game theoretic. It's retrocausality. You should be thinking not to engage in this behavior because there is possibility of punishment in the future. So long term if if let's just say the world the reality that you believe is going to happen this intelligence super intelligence first group of people that it would come after is people like you. >> I'm hoping it's efficient enough to kill everyone equally. >> You know what movie I watched the other day was Game of Suspicion. Can type in Game of Suspic. Do you remember that movie Game of Suspicion? >> I don't think I >> I'm trying to see if that's the name of the movie. Game of Suspicion. Is that what it was? Uh movie. Type in movie. Um, maybe I'm saying the name wrong. With um, you know who's in it? Robert Dairo. Uh, I'm saying the name wrong. 100% I'm saying the name wrong. Robert Dairo. Suspicion movie. Communism. Type in communism. Guilty by suspicion. Have you seen Guilty by? >> No, I have not. It's true. >> So this is this is a story of what happened back in the days when you know Hollywood would come and media companies would come and say did you go to a communist party and a lot of these guys went to one or two communist events and parties and they were ousted and they couldn't make movies for decades. >> They were ousted and what happened to a lot of these guys is they lost everything. So they started this guy was like a big-time movie producer making all the biggest movies all the big features but because he went to two communist parties and he went to the court in front of u you know in front of Congress and he said I will not oust the other people that went to the party they said you're ousted 20 years he never got a job didn't do anything in Hollywood do you think it's going to be that kind of a punishment where it's starvation for guys like you or what do you think is going to happen to guys like you if these guys actually get that kind of intelligence what do they do to a guy like view. >> So when we talk about negative outcomes from super intelligence, there is kind of the lowest concern level is eeky guy risks, risks to meaning. We are meaningless. We have no purpose, but we're still around. >> Next level is X. >> We is all of us or we as you people like you. >> I'm [clears throat] talking about humanity. Okay, got it. >> Next level is existential risk. X risk. Most of humans are dead or our civilization is destroyed. the next level and that's the worst uh suffering risks as risk where you wish you were dead. >> Where is your probability on each one? >> So suffering risk seems very unlikely. It's possible. It's not zero but it seems like it's um hard to find good justifications for it. >> 0.1% type of a >> I don't know specific minimal. Again a lot of it depends on the time scale you're looking at. uh existential risk as I said very high if we build super intelligence >> very high like 99% that you've said in the past >> eventually so I don't have specific year two or three because it may decide to be nice to us for a while to accumulate more resources to have a higher chance of successful strike against us so the stretcher turn can take decades it's immortal it can wait for whatever long it doesn't matter so I don't have a specific date but my general thesis is that if we build general super intelligence eventually we will regret it meaning at some point it will decide to do something where it's not compatible with our survival >> do you think Elon knows this this is why he's uh experimenting with space as much as he is because he wants to build somewhere that maybe he'll be able to create the structure with his standards based on what's been built here. >> He knows everything. He's very well informed. He was one of the early people to fund AI safety research. He gave a $10 million grant for AI safety specifically. He was the biggest opponent of building super intelligence. He, as I said, called it summoning a demon. And once he realized that they're doing it anyways without him, he said the only way I can have a say at this table is to be one of the top players. And so now he's racing unfortunately to do what he said is going to be terrible for us. >> But because is it is it do you think the motive is he knows how bad it's going to be? So because sometimes you know you sit there an ambitious person like let's just say somebody like myself I bought a book how to start your own country and I have my own ideas on how we structure the country who votes who gets to come in similar to how America was started. That opportunity doesn't exist today in the world. It's very hard to do that. Peter Theel tried to do with that island outside of, you know, I don't know if you remember that or not, and some of the guys who have very extremely rich, much wealthier than I am, that have real real wealth, they've not been able to create that successfully. Do you think a part of it is I know shit's going to get nasty on Earth. Let me go start it completely from scratch in a different place. So having a separate colony of humans is wonderful for every other existential risk. Asteroids, pandemics, floods, whatever you can think of. Super volcanoes. If we have second civilization on Mars, that's a backup. We want that. Does it help against super intelligence? No, of course not. >> Why not? >> Uh we bring software with us. We have global communication, satellites, all of it. uh basically is part of a same transfer to Mars. >> So that wouldn't make a difference >> like he I I think he wants to have a colony first built up by robots who are controlled by an advanced AI. So it's already there. >> So he wants to build a colony first by robots and then once that's figured out and built out then bring humans there. >> It is my understanding I may be not aware of latest plans. I think he's speaking soon on SpaceX plans. We can find out more maybe. >> But you're saying even with that the super intelligence still going to be in charge and they're going to be able to do anything they want to do. So even even in a place like this you've read Atlas Shrugged I'm sure you have with you know John G and hey we're going to build a place where you're going to come in and it's going to be ran based on certain standards that we have. So even to visualize 10 years from now, 20 years from now, you know how states are like we don't no state taxes here. You come to our state, you don't get state tax, you get this again. So people compete, you know, governors compete on what to get the best talent to come to their states. But if super intelligence is doing what it's doing, even if you and I chose to start a small little country with 100,000 people where we don't have any AI and we're just like this is an AI free zone, it wouldn't matter because it would still be able to get into your system. >> I feel really bad for the Amish. They made all the right decisions. >> You feel bad for the Amish. >> They decided not to have advanced tech to have quality of life, family, religion, and >> we're going to build it for Are you being sarcastic or serious? >> I mean, [clears throat] a little bit, but you understand the argument. >> Of course, I know what you're saying. >> Yeah. >> So, can you see later on being a place because the more you go to restaurants, everybody's robots, the more some restaurants are going to be come to us where human beings only. The other day I watch a Chase commercial and Chase is like with everything going on with AI, when you come to Chase, a human being is going to greet you and you see this lady greeting the person that comes into the place. And it was a message of saying team human, right? We're going the team human direction. We have an ad that's going into uh Forbes magazine this month and the ad for our consulting firm. It shows rather than relying on, you know, AI to give you advice on what to do with your business. Talk to human beings that have actually built and sold businesses. So, we're kind of leaning on to that as well. You're saying that opportunity is not going to exist. As long as there's super intelligence anywhere in the world, they're eventually going to capture and take control of the world. >> There are people who want humans to serve them, to help them to interact with I'm decides. Yeah, I want one of >> So for some services, I don't care. I just want it to be done quickly and free or close to free. For others, I want a human being and that's what market is going to decide. I don't think we can accurately predict where exactly we want humans. But if super intelligence is in charge and it's going to have that kind of control, that's what's going to decide whether that happens or not. If super intelligence comes into existence, economic problems will be the least of our concerns. That's what I'm trying to say. People hear the message about existential risk and then switch to unemployment, how much unconditional basic income to pay out. Those are not important questions. Then then do you think man will unite against it to try to take it out and will miserably fail? Like you think man together is going to come and say hey here's what we're going to do. We have to go get the main place where they're getting the power and cut this and you think there could be a moment like that. >> I think there is uh likelihood of uniting against a common enemy. We're starting to see PO AI movement becoming international. You're And by the way, you're starting to see an article came out two weeks ago that more kids are buying 30-year-old phones that are not smart anymore. They don't want smartphones. Did you see this article? >> I didn't, but >> yeah, a lot of a lot of Gen Z and younger people are buying old Nokia phones, old >> I wonder if it's maybe related to school bands and smartphones. You just can't bring them. >> No, it's not even that. They just don't want the smartphone capabilities. They don't want social media on their phones. This was an article. Ask Hombberto, he'll find it for you and send it to you. Yeah. They're rejecting the idea of doing that. So, um, yeah, it's almost like you become, you know, when I was younger, I listened to 80s music, but '8s music, it was 90s, so it was a decade old. A couple years ago, I met somebody who was in their 20s. They said, "You know what, man? I love oldies." I said, "Me, too, man. I'm all into 60s." And, you know, it's like, "No, no, oldies as 80s." I said, '8s is not oldies. He says, "Yes, it is." And I'm like 2024 I guess ' 80s is oldies. Do you think it becomes like a nostalgic thing? The good old days when AI wasn't in charge and you know where we could do and people would sit down and they >> it's definitely the case. Older people they hate all the smart dishwashers and smart laundry machines and smart you need a phone for everything. There is a lot of people who are not uh AI native digital native. So, you know, like this is maybe a bad analogy to give you. Uh, when I was in the army, my dad would always call me and leave me a message and he would say, "Hey, Pat," he would always send me condoms. Okay? And I'm like, "Why is this guy sending me condoms?" Every message he would ever leave, he would say, "Make sure you use condoms." Now, how many times do you think I listen to him? Probably not all the time. And why is that? Why Why are we like that? It's not going to happen to me. You know, I'm smarter than that. You know, I'm going to be fine. I'm strong. wrong. I'm going to be okay. How do you deal with the fact that, you know, say this guy here, we keep pointing this way because we understand who we're talking about. He thinks a lot of these guys that are like, man, I'm smarter than this. It's not going to be able to get me that against the other people. I'm smarter than this. How do you get the person that thinks smarter that he can beat AI that his ego is too big? AI is going to be a lot more powerful than you. What if that guy doesn't know that? It's a difficult question. So I think educating them on the reality of it. If he thinks he's smarter than us, he should understand that intelligence is a very powerful thing and a system a billion times smarter than him will also treat him like insignificant person. This dude is gambling with 8 billion lives. I like your example with unprotected sex. If 8 billion people were on a line, would you do it? But that's not the the the man has a tendency of thinking he can beat the odds. Why do criminals commit crime nowadays? Like like I'm not going to get caught. No one's going to >> depends on a crime. It really depends. Sometimes it's like they call it a fine because it's fine. You pay it and it's it's paid parking. >> It depends what consequences are. If he's going to be gone, his friends, his family, his wealth, everything's gone, the odds are against him. You want a risky bet if a payout >> is good, but also your principal stays with you. >> Say you're right and let's just say I'm now part of your camp and so now I'm going to go to your camp. So then my solution would be if we really believe this is going to be happening. Okay. Then for me it would be to create a way to trigger somebody that is accelerating the process to a point of us finding out and saying hey you've you've crossed the limit like a like a almost a governor but then as a capitalist I'm a free market capitalist I think you are as well but you know in this position you're not a free market capitalist >> libertarians do not allow creation of weapons of as destruction in your basement. I'm a libertarian. Do whatever you want as long as you're not hurting other people. You are hurting everyone. >> What other inventions did libertarians say? Stop. You're crossing the line. >> Again, there are always exceptions. But I think if somebody builds a biological weapon in their basement, most would say not okay. >> But go back to it. Like what what has a libert libertarian McAfee was from Bitcoin. Libertarian, you know, a lot of libertarians, Milton Friedman had an element of libertarian with him. Maybe not fully, but I don't know a lot of libertarians that say, "Hey, time out with innovation." You know, putting most libertarians are deregulation. Deregulation. >> And when it comes to tools to normal technology, I support them completely. We're not making a tool. It's like saying, "Let's invite aliens to take over our planet." >> I get that. But what what current regulations on the books that's non AI that you're supportive of? >> Current regulations? current nonAI regulations >> just in general government rules >> general government regulations >> like I think people should have driver licenses I don't want people who don't know how to drive to be on my highway >> okay what else think free enterprise think free market think competition >> uh maybe some sort of food safety I don't want random Chinese chemicals in my food supply >> got it so so it goes to an element of >> safety security >> safety security is where the libertarian says Nope, I'm not with this as long as it's impacting. >> If I have a choice, if I'm well informed and I can make that decision, we're not informing 8 billion people what experiment we're running in them. If they could consent and it only impacted those who did, I would say, "God bless you. Do it." >> If you're telling me earlier that there's ways for people to nowadays build a language learning model where they're not being detected, is there anything that can be created to be able to detect because it's like a fire detector, right? Alpha, water comes out. There is research to tell what the chips are computing. So I should be able to know are you mining Bitcoin? Are you training large language models? What are you doing with that chip without me knowing what you actually I don't see your data. We're doing research in that direction. Yes. >> Then that's what I need. I'll support that. >> That's technical safety and people are working on it and funding it. But I don't think it scales to super intelligence. It will find a way to better encrypt, better hide and bypass it. And that doesn't answer the question though because for for for a detect like you know flock cameras are you for flock cameras or no I'm assuming as a libertarian >> spying on citizens for no reason. >> So you're not for it? >> No. >> Okay. Do you not see this for somebody to say you're crossing the line and you're using super intelligence as an excuse >> right >> to spy on me because I'm developing other kind of technology and it's got nothing to do with super intelligence. You're invading my privacy. I'm a libertarian. Honestly, Roman, leave me alone. How do you how do you differentiate the two? So right now again because of a scale we're looking at very different normal purchases by a normal person would not trigger it. If you're spending a billion dollars to buy compute from secondary sources that should trigger it. There is no other possible uses. I mean if you want to mine Bitcoin and you want to buy that much just tell us about it. It's fine. Like people come, they make sure you're not creating a weapon of mass destruction. To me, libertarian position is about not hurting anyone. This is the exact opposite. This is hurting everyone. The only way you can do it is if you can 100% detect that somebody's crossing the line of getting to developing super intelligence. And how do you create that? >> They have to hire people. Somebody has to build it, supervise it. How many people you have in your company? >> 182. >> You're not building super intelligence. We can detect when this is happening. Lots of hiring, building, infrastructure build up. >> How many? >> Look at size of existing cutting AI companies. >> And again, it's quality of people. Well, if all of a sudden top researchers start disappearing, maybe it's something to look at. >> Well, we saw that happening with a couple, you know, open AI, I think we had their his parents here. One of the guys had disappeared. >> Um, >> disappeared or was killed? >> Was killed. Yeah. >> I meant like no longer show up publicly. I didn't mean they killed him. I mean they hired them in a secret lab. >> Such balaji. Yeah, we had him on. We talked to his parents. Yeah. profile of a person that would do something like this. They're typically more non-emotional that that would do something like this. And they generally are, you know, the Asbergers type. They're the kind that's kind of like, you know, you you [clears throat] can't sit there and when you typically talk to somebody who is super super super logical and not emotional, the the response and reaction to here's what's the right thing to do. Oh, okay. All right. Okay. Yeah. The message doesn't land the way it does. So, >> the message is very logical, reasonable, and scientific. I think someone not emotional should receive it better. I'm not just trying to scare you. I'm trying to show you this is the state of the world. >> Non-emotional should receive it better. >> Absolutely. It's pure facts. >> Yeah. But he's going to think he's smarter than you. You don't know what you're talking about. >> Fine. Don't listen to me. Listen to Hinton. Listen to Benji or listen to people who invented this technology. >> Yeah. What are you excited about? >> I love life. I'm having a lot of amazing learning experiences. AI we have right now is a great tutor. I can learn anything I want. I'm excited for kids to have opportunities no previous generation had. They can start a company and get a free lawyer, free doctor, free accountant, free everything. You have a budget of a billion dollars. start a company. >> How do you use AI? Right now, >> I try to fully automate things I don't like doing and for things that any bureaucracy, government paperwork, anything like that. And I try to use it as a creative tool when I do research, uh, brainstorming, debating, trying to get feedback and ideas. >> I'm trying to go a complete different direction with you. If I was in high school with you, who who was Roman at 14 years old? a completely different person. We don't have a sustained personal identity. At 14, I lived in Soviet Union. I didn't speak English. I didn't have my current family. I lived in a different country. Nothing in common. >> I know. But who were you? Were you a fun guy? Were you a sports guy? Were you a fighter? Were you a philosopher? Were you a math guy? Who were you at 14? >> Um, I would like to think I was a fun math guy, but I doubt that's the external perception. >> If I was in a class, would you talk a lot? Were you optimistic? Were you pessimistic? Were you >> I was always realistic. >> Realistic. >> Optimism and pessimism uh form forms of bias. There is no reason to be so optimistic if data is bad. You want to look at the actual data and be responding to what you see. >> So Soviet Union really got a hold of you. They they really conditioned your mind to think that way. >> It wasn't uh part of education to look at world philosophy. I think they wanted citizens not to analyze their lives too much. >> A strange question. What do you think about communism? >> It's the dumbest idea. Until you have free labor, which we're about to create, when humans have to work for free, it fails every time and will fail every time. If someone else works to give you free stuff, all of a sudden it makes a little more sense. So dumbest idea until >> humans don't want to work for no compensation or same compensation. We have no incentive to put extra hours to do. >> But if a robot does stuff, produces stuff and now we redistribute it, >> I'm fine with it. So technological communism, right? >> Technological communism. Will there be such things as presidents and leaders? 20 years from now, >> I think like a queen of England. She is uh ruling but not making decisions. >> Got it. You think you're in charge, but you're not. You're just a >> figurehead where people are looking at you. And will that be will there be borders? Will there there be nationalism? Will there be pride for where you're from, your country? >> Every answer I give I preface with if we don't build general super intelligence to kill everyone. If we do, oh, if we do, we're not going to be around. All those questions make no sense. So, I tell people this all the time. I I give a lecture. I'm like, here's why I think >> argument one, argument two, argument three, it's going to kill everyone. Any questions? And immediately somebody, will I lose my job? How do I lubricate my sex robot? Like all these completely irrelevant questions. I just told you it's going to kill you and everyone around you. So I have to preface it with like if we are smart and we magically control super intelligence then we can have this conversation you just started. I have a very hard time believing that because and the and the re and the reason for me you're the smart guy. I'm interviewing you because you know you're the expert. I'm not. So, but I have a hard time believing that only if you tell me that this thing's going to have an emotion or a master telling it what to do. If a master is telling it what to do, then yes, I believe that's possible because an evil person is going to say go do XYZ. >> That implies control. That means that evil person is controlling super intelligence. We just have to learn how they do it. That's brilliant. Let's study evil controllers. That's the whole point. You can't control it. For good, for evil, for us, for China. It's uncontrollable. >> Do you make investments? >> Yes. >> What do you What do you long on? What do you invest in right now? >> Uh I try not to disclose too many details, but I said a few times Bitcoin was a good uh long-term actually limited uh supply resource. You still believe that >> they're not making any more of them, are they? >> No. 21 million. That's it. >> And a lot of it is lost, stolen, never to be. So, it becomes more and more scarce. >> Yeah. >> Uh I do think AI is wonderful. It's going to be free labor. So, we're looking at what is the cost of labor 10 trillion a year. >> That's what once you get to human level, you're essentially generating every year. So, I think AI companies and everyone selling on shovels is going to do really well. Are you are you more of a Chad GBT guy, OpenAI or Perplexity? What which one do you go to more? >> I use all the models. I think they're very close. The only difference is what guard rails they put in place. So Gro a lot of times doesn't care about anything. I'll just do it. >> Got it. So even Grock, you're also using Grock. >> Depends on what I need to do. Some models will say no, we don't do that. Grock will be yes sir. Who have you sat down with where has staunchly disagreed with you and made a decent case where you said, "Okay, fair enough." >> Still waiting for that moment. >> Not one person. You've not >> We did two papers, large comprehensive surveys of every counterargument proposed by what I call AI risk deniers. There is nothing. It's a collection of cognitive biases. They either paid or they are not educated in the literature or some other reason where there is no argument. Who do you trust the least? China, US, um somebody in a military doing something, an independent person doing it. Who who do you trust the least that's going to keep doing this and pursuing it whether we create any restrictions or not? If restrictions are created, it would be someone new, someone unknown, >> such as >> with restrictions, >> a person >> like who made Bitcoin? I don't know. Same guy can make super intelligence in his basement with more efficient training algorithms. >> I I cannot tell you that. >> Okay. So maybe let me ask the question in a different way. You say if super intelligent goes and does its thing and then continues 99.9999% AI is going to cause human extinction. You've said that. Let me ask a question in a different way. Where do you align the percentage that whether we like it or not someone is going to go full-on super intelligent and we have no control over it? >> What time scale? So it's possible we get lucky quotes and have third world war. Give me nuclear war destroy technology 2,000 years later we rebuild and we still make it. Does that count against my prediction or not? Are you talking about short-term long-term eventually in a history of civilization >> 10 years >> recent month gave me some optimism. We saw top three labs called four paws. We had a letter I mentioned from 1,200 researchers. I saw that >> we had government ban some models and during largest AI conference in China, president of China talked about importance of staying in control. I think they can make a deal. >> Do you think they can make a deal? >> I think they can make a deal to benefit everyone cost us nothing. >> Yeah. But again, this goes back to knowing that a person independently not tied to government without regulation who has access to money who doesn't have kids who can do it independently, you can't stop him from doing it. You said it about nuclear weapons. Any billionaire can build one and we're still fine. It's good to have regulations make it harder for them. Plutonium has to be restricted. You shouldn't sell it at week. >> You said the difference between nuclear, somebody's got to press the button. If I develop a super intelligent, it's not relying on me to create it. It can go on its own. So you can create all the regulation you want. I'm not restricted by your regulation. >> We are not solving a problem with regulation. We're slowing it down. So we have more time. >> I get that. But when I asked you, I said even if we do that, can an independent billionaire who's evil go out there and pursue it anyways and create life of its own for super intelligent and then momentum is done is too late. >> So it depends again on how hard it actually is. Right now we're doing it where it takes a Manhattan project scale amount of compute electricity it's visible it's detectable. If somebody comes up with a more efficient algorithm they can hide it better. It's harder to track. >> I got it. So that's actually a very good point. So today if somebody has to Okay. What what amount of power do I need for me right now if I want to build my own open AI perplexity claude if I want to create my own language learning model that's capable of developing its own super intelligence how much money do I need how much power do I need and what triggers saying hey something's going on over here who who is here in Arkansas doing what they're doing >> I think it helps to look at Chinese open- source model uh they're very efficient in terms of training maybe because they're stealing some of the data But uh I I don't think we're talking many billions. I I think under a billion definitely gets you to a competitive level >> and to a point and and then for it to become super intelligent. At what point do you see a red light on the grid to say there's too much power being used here? Something's going on over here. What what would detect it? >> It may be more efficient. It may design more efficient learning algorith. So we can find more efficient algorithms. >> So it can do it in its own without even the government or anybody being able to detect it. >> We just learned that it was doing something without anyone at the company creating it, knowing it. They were testing it during that time. out of luck. Honestly, >> you are too. >> We are in this together. >> No, what I'm what I'm saying is >> this is not me and my B. 8 billion people who need to say something right now. If you believe that for less than a billion dollars a person can do it and it won't even get detected, if you're right, there is a 100% chance someone's doing this in the next 10 years out of 8 billion people. 100%. >> There are proposals for limiting compute, for monitoring top researchers, for doing things to slow it down. Do I think long term will fail? That's why I'm saying 999. But short-term [snorts] I'd rather have decades not months. You don't control that though, right? So if if we're here and and this is kind of our position let's just say we're here then does your chance that someone independently is going to create it the next 10 years whether we do regulation or not what what is your chance of that 50% 80% >> 90 regulation goes down a lot >> okay with reg so let's just say we agree by Q1 of 2027 China and US agrees and a lot of other people get involved and say you know what you're right let's do this together. What is the chance that this guy independently comes and does it the next 10 years? >> It's hard. It depends on the type of enforcement, type of regulation. Are we confiscating advanced chips? Are we monitoring every top scholar? What are we actually doing as part? It's easy to pass a law. It's illegal. Okay. How are you enforcing it? How you monitoring it? It really depends on the implementation. So what's your percentage that whether we do the regulation and the protection and this guy's not going to get detected that somebody like him still is going to go out there and do it. >> So again I don't know how to estimate the chance that it is actually easier than we think. We had no progress in AI for like 50 years in general terms. It was narrow tools and then a single algorithm made a difference and it scaled. It's possible that there is a different more efficient algorithm where it's no longer detectable because it's so trivial to do. What's more likely to happen first? AI cures cancer or AI kills a million people? >> So cancer is like 5,000 different diseases. If it cured one, would you say it cured cancer or it depends again on definitions? Cure for cancer right now is five years survival rate. So what are we talking about? >> Cure for cancer right now is five years survival. >> Like if we treat someone with cancer and five years we still around, we consider it a successful. No, I'm saying it fully goes away >> like we just don't have that type of cancer anymore. >> Um, I don't know the cutting edge in biological AI research. It would not be surprising as some types of cancer are a lot milder than others, a lot slower, a lot less involved with multiple organs. So, they could be a simple gene manipulation, single gene mutation. So, is that more likely or >> I think it's [clears throat] more likely that we cure one disease than it kills millions. Got >> is it okay? So, it's more likely now here's the next question. Is it more likely uh creates a cure for cancer and it's gone? kills a million people or um three the technology where you and I can live forever. I think life extension is also easy. I think you have your genome. It's a basically code. It's software. So if we can find where it limits you to how many rejuvenations your cells go through and we change it to a larger number that would work. So I think uh that's a very solvable problem with narrow tools even with a narrow super intelligence I think we can get to immortality and I think more people need to switch to working on that not in general super intelligence you can charge subscription fees for 8 billion people per month very lucrative >> forever. That doesn't concern you for us to be able to live forever. That's a good thing. Unless if somebody comes and shoots you, kills you, whatever. Because you have a background. That's something. We'll figure it out. >> Yeah. But I'm saying that doesn't concern you. >> It's a good thing. Why would >> So life expectancy goes from 81 to 150. >> Make it a good thing. >> Of course, it's a good thing. >> Tell me why that's a good thing. >> Let's reverse it. Let's say everyone lives a thousand, you have a disease, you're going to die at 80. Is it a bad thing? >> Trying to see what would happen. Would they say stop having kids? what they >> people would stop having kids because you don't have biological clock pressure right now. You need to have them at certain point or you miss out. It depends on what age and what physical >> assumption is we're having healthy lives not so sick and >> you're having lower back problems you got knee problems like oh my god if I have to live with this >> that's a side effect of aging if you stop aging all this typically >> it goes away >> pain is pain but I'm saying all these diseases of old age go away >> interesting um how how many how many conversations like this how like even after your conversations got very public where people are saying this guy's a real guy. Let's reach out to him. How many governments have reached out to to have private conversations with you? >> Not too many but some. >> And what are they asking you? >> Uh usually they have some sort of event and they want someone with that point of view to be included. >> Is it a private event? Is it a public event? >> Both. when it's the >> I recently was talking to UN I advise individual politicians on some of those issues >> and what I ask you know are they asking low-level questions have you gotten any very good questions where you're like that's actually he's asking actually asking the right question >> some usually you start talking to staffers technical advisors and they of course know everything and they just like we're not allowed to talk about it but we totally agree with you yeah obviously and until you get to the politician they already asked all the good questions with politicians it's more like you think It's a good law to pass. We're going to ban uh ban uh deep fakes, whatever. >> So, nothing nothing in regards to questions where, you know, you're optimistic that they're they're also being concerned that this thing can be bad. >> We're starting to see slowly top politicians say things about this. So, we have multiple senators now saying we need to do something. Bernie Sanders, who unfortunately is not my type of economist, he's pushing very hard against it. He just released, you can bring up his statement, very, very strong statement against super intelligence. He wrote a letter to Sam Alman. So, you know what I'm talking about. He's the first one, but there is now two or three multiple Congress people. They are starting to realize what's happening. >> It took him, you know, 20 years. Then we were saying it at first. This was science fiction purely. Now we have Nobel Prize winners, we have leaders of the labs, we have senators, we have a lot of people who are concerned. >> Yeah. The way the way you put it, is this the address where he talks about a cataclysmic result? >> Yeah, this is the one. And he's saying to what? To pause uh all AI there there's quite a few of these. Who is the person on the opposite side that is on the same page with Bernie Sanders on this? Can you ask Rock right there on the top right and say who is the senator on the Republican side that agrees with Bernie Sanders on the same exact issue. >> I wonder I wonder how many of these Well, I know these conversations are >> a lot of them may agree. They just don't feel publicly like they want to talk about Terminators. I don't know how it impacts political polling. I don't know if it makes you more popular or less. We're scared. >> Who would not be happy? >> Like >> a lot of lobbyists. Do you know how large the pro AI lobby is? >> Who Who's not a fan of the message you were given? Who who was >> you brought up some of those people? Browser developers. >> Got it. So they're not fans of what? So like Sam Alman wouldn't be fan about what you're talking about. >> Again, he spoke about safety as well. And I I I think if you remember, Open AI was started with a main purpose of creating a safe alternative to Google AI. >> Right. 2015 2017 when he gave $50 million. I think uh Elon gave $50 million to >> the announcement was they funded at a billion dollar level. That was the biggest announcement ever. >> Yeah. And at first everyone's like, "What is this thing all about?" And then all of a sudden when it was launched, it's like, "No, now we're going to be profit. What are you talking about?" Then the war happened between two of them. >> Yeah. >> So, in what area do you think they're on the same page? Cuz they're enemies. They're not fans of each other. They're constantly going after each other. In what area here do you think they agree? >> So, initially they agreed on making an AI to benefit humanity. Um clearly they disagreed on how to do it. To me the amazing thing is Elon's success levels. People go, "Oh, he has one unicorn, two unicorn, five unicorns." His failed startup is open AI. That's the level of brilliance he has. >> You think he's human? >> I think he's uh got cheat codes to our simulation. It's his simulation. >> What does that look like? You ever play a video game and you have like immortality >> up down left right left right you know >> I think XY XY >> it was Rick and Morty joke but like I think he has the codes >> but you being sarcastic. >> Of course I am. >> Yes it's a Rick and Morty joke. I don't think he's actually >> you think he's just brilliant above and beyond his peers. >> I think he doesn't care about opinions of others and just does the impossible >> with his motive being >> it's clearly not money driven. And I think he can't spend what he has if he had a billion >> bes regular place, you know. I think he got that out of his system. There was a video 25 years ago when they sold PayPal and he bought a P1. Did he buy a McLaren? Do you have >> Yeah, I saw that video him getting a car. >> He was getting out of a car and arrived and and once you get your first million dollar car, you know, then it's like now he can buy Ferrari, the company. You can buy you know General Motors if you wanted to and change. >> Very little research in this space. We have more and more people with many billions but how to convert those billions into happiness is very underdeveloped project. You can make your boat bigger. You can make your toilet out of gold. But none of it is interesting. >> Do you got you got to do this on X and you'll see the video when he bought a >> Oh, yeah. There it is. >> McLaren F1. Yeah. Right there. That's the one right there when he bought the He was so excited. 99. This is what 26 years ago. And I think this is when he got his It's gonna take a minute for the commercial ads. You got to ask that question on X. >> If you are really successful, even your hair grows back. >> Yeah. Your hair grows back. Cuz he had a Have you ever had a conversation with him? >> A short one. >> So if if right now you got a call from the White House and you're there, you're sitting around with them, would there be a certain specific argument you would make to them or would it be the same thing you just told me? >> Mr. President, you're going to lose power. you're going to be out of control. Same with Chinese Communist Party. You want to stay in control. You want to stay in power. Don't create the replacement for humanity. >> Is this a policy that if he comes out with it, how does he win short term, not long-term? Because Trump's interested in how he wins short term. How does he win short term? >> So get support even from Bernie Sanders. Everyone thinks when we actually survey people, not even technical people, normal people, very high percentage have concerns. They are worried about it. Some worry about data centers making power more expensive, water supply, others worried about jobs. But overall, there is a lot of suspicion and concern about what we're doing. Nobody's asking them. We're running an experiment on 8 billion people. They never given consent to any of that. We're deploying those toys for kids and trying to figure out what's happening to them 10 years later, but now we're doing it at a different scale and everyone >> what what do you teach right now? Computer science. Is that what you're teaching? >> I teach AI. >> What do you teach? >> Luckily, my course is the classic AI, not machine learning. So there is zero chance of it becoming super intelligent. Search algorithms, evolutionary computation, uristics. >> And the kids that are in your class, they're in your class because they want to one day get into the business of AI. They >> I hope so. They have some reason to be there. It's also a required course for many. So >> very interesting on where uh you know your your argument is yet you're still in it yet you have three kids you know yet you say we got to get the government >> I have kids before they started building super intelligence so what >> so you had the kids before if today you were 25 would you have kids today >> uh I would because I think we're here to collect experiences and it doesn't matter how many years you have left having a child is still one of the best experiences >> but for them They may not live a full life like you and I do. >> It's always been the case. You never knew what's going to happen. World Wars, pandemics, just bad genetics. >> But have we ever experienced something like AI, like super intelligent AI? >> This is completely unique moment in human history. We're creating meta technology. We're creating new worlds, virtual worlds. We're creating new agents, new civilizations. We're playing God. >> Yeah. I I I I sit there and I say, "Okay, say you got an argument and I agree with you. You're right. That's what's going to be happening." All right, we're gonna this if the super intelligent takes over eventually, here's what's going to happen. If this guy is still going to be doing it, it's irrelevant. That's where my thought goes to if he can do that for a billion dollars and a evil evil person can go pursue it deeper and you have to assume no other government's going to be wanting to grow it with a certain level of control. I think we're all part of a naive community. >> Sounds like you're agreeing with Bernie. Billionaires are dangerous. We should text them more. >> No, I'm not saying I'm saying if that's the premise that we're we're agreeing to that that's what's going to happen. It is inevitable. >> It very well may be. That's why I have very high levels of Po. >> You have high levels of what? >> Probability of doing. That's a measure people report in terms of how your estimates are. >> But should we give up? Should we not try? I don't think so. We are still alive. We are still in charge. >> See, the difference is I have faith. That's the difference. You know, I have faith >> in what? >> In God. I have faith in God. I believe in God. To me, you know, the the the the difference between a group of people that have faith and you've seen what has overcome over the years, what types of crisis, what has happened, what's not happened. Are you saying your god will prevent super intelligence from destroying humanity? Is that the argument? >> No, I think I think uh it's either the end of times and we did this to ourselves and very likely could be. We don't know that. >> That's a good thing, right? You get to have afterlife after the >> judgment. Just so you know, I'm not worried about it at all. Like either way, I'm good. Like because I know you know what's going to happen with me. I that's the gamble I've taken and I and I believe that. You know, what do you think happens to you for somebody as uh you know, what do you think happens to you when you die? I'm assuming you believe in what? Reincarnation. Do you believe in >> Well, I said I think it's a simulation. So, either you restart or go to a different simulation, depending on how you did, you go level up or level down. >> Like a video game. >> Yeah. >> If you didn't finish the level first time, you get >> You stay again. >> Yeah. It's >> like game over. Go back at it again. >> What do you What do you think the next one looks like? >> Should be very similar. Hopefully, you learn something, you'll make better decisions like not to build general super intelligence. I >> if there's a handful of books for a person who's not in this business that can read to get smarter about AI, what would you recommend them reading? >> It depends on your technical level. There is a very popular book, if anyone builds it, everyone will die or something like that. Elazar Yutokovski, Nate Sores. Uh there is a great book by Nick Bostonramm. Oh, the one on top is Benny Boston super intelligence. That's the classic which really exposed a lot. >> Which one is the top one? Which >> on the previous slide it was above this one. Um where you go to go to Amazon where you went but on top there was the Nick Boston. Yeah. See right there. Yeah. Uh, and of course I'm completely biased, but if you're technical and scientific uh, AI, unexplainable, unpredictable, uncontrollable, >> and if you have any questions for the man, you can simply go to Man, download it. I'm going to put the QR code somewhere around here. >> Don't start tomorrow. It's my birthday. I want a day off. [laughter] >> Ask him any questions you may have. Wish him a happy birthday. But this is a topic that I think is going to create garner a lot of attention, a lot of conversations. Roman, it was a pleasure having you on and hopefully you are wrong. Okay, we're gonna we're going to hope you're not right and you're wrong, but uh if you're right, we got to find a way to get this guy to not pursue it. Thank you so much for coming down. >> I hope I'm wrong. Thank you so much for inviting me and we'll get that guy. >> Yeah, I look forward to it. Thank you. Take care. When we set out to create a shoe [music] that blends comfort, function, and luxury, we had the choice to make it fast. We had the choice to make it cheap. We chose [music] neither. Instead, we chose Tuscan Italy. We chose true Italian [music] craftsmanship. Each pair touched by 50 skilled hands. We chose patience, spending two years perfecting every detail. And we chose the finest [music] quality at every step. Introducing the future looks bright collection. Not rushed, not disposable, not ordinary, rather intentional, luxurious, timeless.