Should autonomous drones, known as 'killer robots,' be banned in warfare?

With technology advancing at light speed, we are seeing it before our very eyes. But with the rise of AI, there is a great fear over the use of killer robots, autonomous weapons used in the battlefield. But do the pros outweigh the cons? Well, we are going to have a debate tonight. First, we're going to introduce our panelists for this topic. We have Stuart Russell, computer science professor at the University of California, Berkeley. also the president of the International Association for Safe and Ethical AI. He is in Paris. And in Washington, Professor Jesse Kurpatre, the director of the Mason Autonomy and Robotics Center at George Mason University. Also in Washington, Jake Steckler, research scholar at Govai. But before we get started, here is some background into the topic of killer robots. Technology is moving rapidly [music] and international bodies are playing catch-up. The UN and the Red Cross have jointly pressed governments to set clear international limits on autonomous weapons, warning that the technology is moving [music] faster than the rules. They say the dangers have grown since their last appeal 3 years ago, even though they have not confirmed these systems being used in combat. Rather than demanding a total ban, they're [music] calling for targeted restrictions on how such weapons can be developed and deployed. Advocates [music] back the push because they fear machines could soon decide who lives or dies without meaningful human oversight, while some states and defense sectors oppose tight rules, arguing they could hamper military innovation and weaken strategic options. But is it humane [music] to use killer robots with little human oversight? Or is all fair in love and war? Does it help preserve life or is it a threat to it? Should autonomous drones be used in warfare? >> All right. And now we are officially ready to begin. We're going to start with the first question. You each have 30 seconds to answer. We're going to go in order of how we introduced you. Starting with Professor um Russell. The first question is, should autonomous drones be used in warfare? Go for it. >> I think the answer is essentially no. Um, we have to ban particularly small anti-personnel weapons that can be launched in the thousands or even millions by a single person pushing a button. Those are weapons of mass destruction. They are much cheaper, easier to use, easier to proliferate than nuclear weapons, and they could make the world completely unlivable for everybody. >> Wow, that's a scary thought. Okay, Professor Kurpatre, what are your thoughts? >> Yeah, I mean, I really think the answer is it depends, right? Autonomy is a really broad category. The way that it's been cast with the kind of killer robots camp has been way too broad. Some applications may in fact uh improve precision, saving civilians and soldiers alike. But what we should do is that we should strictly govern autonomous weapon systems. We need robust test and assurance, legal review, human control, and accountability. >> All right, Jake, your take. >> Yeah, I think efforts to ban autonomous weapons are generally misguided for a few different reasons. The first was hit on by the previous guest is that the definitions are a little unclear. We have autonomous weapons today that don't pose the sort of risks that we're talking about here. And as a weapon systems have gotten more advanced over time, they have reduced civilian casualties. And last, I'm a little skeptical of the feasibility of enforcing a ban and don't think it's directly analogous to uh different types of arms control. So my basic position is that these things are inevitable and it's a matter of the accountability and safeguards that we put in place. >> Right. And clearly there's no stopping the development of these. So whether it's ethical or not, it's you look at where where we're going and how fast AI uh is developing. But would you say it's uh morally acceptable, I mean to delegate a a lifealtering decision? I imagine mistakes could also happen. And we'll start. Oh, sorry. We're going to go to uh Stuart again first. >> Well, so notice that I did not make a moral argument in my opening statement. Uh it's just common sense, right? We don't sell nuclear weapons in supermarkets and it's not for moral reasons. It's because it's just not sensible to do that. So if we're going to allow people to buy millions of anti-personnel weapons that they can launch and kill millions of people, we might as well sell nuclear weapons in supermarkets. That's the situation that we're heading towards. So that's not a moral question. I would say however that moral stigma is a very important component for example of our attitudes towards chemical and biological weapons. We used to have very large very advanced development programs to create those weapons and now we would I think almost everywhere in the world feel that it was morally unacceptable [clears throat] to do so and I think the same would help uh in controlling autonomous weapons. And what do you have to say, Professor Kurpatre? >> Yeah, I take a somewhat different approach. I mean, I don't think that there is anything special morally about a human pulling the trigger. Um, and in fact, much of the kind of decision-making authority that we see in life and death decisions and warfare are in fact very much made upstream and the human is primarily pressing a button. So I don't think that there is some sort of magical thing about a machine killing versus a human. >> Right. >> I will say on the >> go ahead. >> I will say on the top I thank you. I will say on the topic of chemical and biological weapons I think that those are very different. Right. I think that we ban those weapons when when the objectionable characteristic is intrinsic to the weapon. Right? So we can think of bio is disease, chemical weapons produce poison. Autonomy produces autonomy, right? It's how they're used, how they're how they're deployed, how they're developed, that is really the meaningful point here, whether it's legally, morally, strategically. >> And uh Jake, from your from your perspective, uh what do you say? >> Yeah, I think I think I generally agree with Professor Kurpatre here. Um I think that uh one thing to add is that enforcing arms control on these things would be incredibly difficult and it is different from these other use cases basically because what we're talking about is is software uh and a lot of it is open- source software and and there are all these AI models from China for example which are all open weight which means that anyone can basically pull them uh off of the internet. Uh so the ability to enforce a sort of ban would be much more difficult and even just detecting that someone has used an autonomous weapon is is also something that's very difficult to prove uh because it doesn't leave any sort of residue behind to show that it was operated by a person versus uh operated autonomously. >> Well, I mean we can see just from >> Can I push back against this? So So obviously it's not software, right? Software doesn't kill people. Uh hardware kills people. Um and so uh what we need to do is to ban the production of autonomous weapons. Uh the software is is a almost completely irrelevant issue. We can put in restrictions on human operated weapons so that they can't easily be converted to fully autonomous operation. Um and uh we can for example uh fix the onboard um computing in uh you know these kinds of weapons so that they can't be reprogrammed. We can we can require that they only operate with special purpose application circuits. Um so there's a number of steps that can be taken to distinguish between autonomous and human operated weapons. We can also require that, you know, if you have uh a million weapons that you claim are human operated, then you have to have a million human operators, right? Um so what we're trying to do is avoid the situation where one person can push a button and kill a million people. If you want to be in that world, be my guest. It's not a world that I want to live in. >> So with that said, I mean, we know AI could make mistakes. Even just using it for a basic search, AI could make mistakes. So how could we even uh you know trust that these that this technology especially if many people are developing it I imagine some technology would be better from one country to another. I mean it seems like a very scary thought from my standpoint. What do you think? >> I mean one way if I can jump in is that is that we can test and assure the way that we have with other weapon systems. Right. I mean, we have a pretty robust regime here in the United States, and I can't speak of other countries that I'm less familiar with of testing and validating systems so that we're assured that they work and they work properly. Now, I will say that I share many of Stuart's concerns. I'm worried about proliferation, right? I'm worried about bad actors. Um, I'm worried about making war easier. I'm worried about um enabling terrorism. I think that where we where we're far apart is that I have a different kind of cure for what I think are the symptoms, right? And I I >> Well, I think that the the symptoms are the sort of worst case scenario where we have a world in which weapons are widely available, that they've been proliferated, that they are used without knowing the the ways in which they work well and where they fail, right? That they're used by bad actors. I mean, there are lots and lots of scenarios in which autonomous weapons can cause concern, right? I I I'm not San Gwen about this, but I also think that a a full ban is neither practical nor desirable. >> Well, and also, I mean, when you're in war, you can't not have soldiers. So, how could AI try potentially replace that at some point when we are having world conflicts and could it get even worse because of it? Well, so I think this was one of the myths that was put about by proponents of autonomous weapons that uh and the phrase they usually use is it it keeps our soldiers out of harm's way. You know, would you rather send a robot or your son to war? Um but what we've seen in Ukraine is that uh the remotely piloted weapons are extremely effective and the death rates for human beings are enormous. There's a there's a fallacy which says, well, we of course are going to use these weapons in the right way and our opponents are not going to have these weapons at all. Obviously, that's a fallacy. If both sides have these weapons, they will use them to kill each other. And that's exactly what's happening in Ukraine, >> right? Yeah. And another thing I want to add, I mean, look, we are seeing so many mass shootings. We just saw something happen uh that we're going to talk about in in in Switzerland. But let's talk about that idea of terrorism. People can find on AI how to build bombs and to do all of these things that no one had access to. Is the same uh can be said when it comes to these types of uh devices? And is there a real threat of random people learning how and causing major catastrophes unpoliced? >> I mean, I think the answer is yes. Wait here. >> Oh, yeah. I'll let I'll let uh Professor Kpatrick go. You haven't gone in a while. >> I mean, I think the answer is yes. I want to go back to the the previous question, though, very very very quickly, which is um Stuart's right, but you know, I think the the idea that um that robots are going to be fighting and doing all the dull and dangerous work and humans aren't. Um is a is a fiction, right? that what we know is that um robots are or you know autonomous systems can be very good at at um finding targets potentially selecting them and engaging them but can't hold territory they can't hold ground or at least not yet right >> and they don't have point >> well yeah they don't have intuition that's right but I think the second point is this is that uh to to your question is that yeah I mean whether whether autonomous weapons are banned or not Right. It is a major concern that any type of new technology, including autonomy and lethal autonomy, is going to get into the hands of bad actors. I mean, that's a that's a major concern, and I can see that fully. >> Well, especially as gun laws and people clearly, it's such a huge um trigger point uh in the US with so many different mass shootings and and gun laws and all that. Is it going to have the bad guys try to find alternative ways to have weapons? Jake, >> I think it depends on whether you whether you allow them. Just Oh, sorry. Very quick. Go ahead. Depends on it depends on whether you allow them to be manufactured in vast quantities or not. So, chemical weapons, for example, uh I'm sure that many terrorists would love to get their hands on them. In fact, we know that they are trying to use AI systems to develop their own chemical weapons. Um but the manufacturers of the of the most dangerous chemicals are required by the chemical weapons convention not to sell them to unauthorized entities. Uh and so that means that they are large the the risk is largely contained and we haven't had any major terrorist chemical attacks. But with these kinds of weapons, they will presumably be as available as Kalashnikov militarygrade automatic weapons. And there are 100 million Kalashnikovs in private hands. So that's the scale of risk that we're looking at if we allow these systems to be manufactured in large numbers. What do you think when we are in real war uh situations clearly as we're seeing with Russia and Ukraine and and what's happened here um locally in the Middle East? How how could having these weapons in more numbers? I mean, could it really lower the amount of casualties or limiting the number of soldiers that may have to go in into a territory like replacing the need for certain units that maybe would be on the ground say in uh southern Lebanon or Gaza for example? I think that uh something that you're seeing in Ukraine is that you're not seeing a reduced number of soldiers in total. I think that there are uh many operators and people behind the scenes working with any of these systems that even do have some autonomy in them. But you are seeing uh a situation where it's created a sort of stalemate uh in this this very large and wide kill zone uh which has basically become a place where no one can move in and and in some ways uh that that could be a way that uh could deter war uh to some extent or even reduce casualties because of the understanding that anything beyond some point is is at such a great risk. Um and you're starting to see in Ukraine uh using uh machines for doing things like um evacuating soldiers uh for delivering goods. So it it does it really depends on the use case, but there are absolutely use cases where you could reduce the the uh potential people that you're putting in harm's way. >> I can tell uh Professor Kpatrick, you want to say something? Looks like you want to say something. >> Yeah. I mean the the goal is right is that no military um wants to get into a fair fight, right? That they want a strategic advantage against their against their enemy. And what we see in the United States is a lot of assumptions about the type of wars that the United States is going to be fighting wrong. Right? The United States has operated under the assumption that wars are going to be short, sharp, and quick and that we would have major technological dominance. And that's just not the case, right? We have nearpeer competitors and and I know that lots of folks want to say, you know, well, we could get a comprehensive ban, but there are actors who are not going to comply with the governance regime that is going to be laid out. Who do I mean? probably China, certainly Russia and you know so a ban is going to give those folks the sort of competitive edge that um that we've seen. So you can imagine the whole world imagine the whole world agrees that they're not going to be lethal autonomous weapon systems just open the door for China to invade Taiwan or for Russia to take Ukraine and hold it without any answer to that question. >> Well and also if it becomes the wild wild west how could it even be policed? I mean, nothing's stopping anybody from sending a autonomous drone over circling over the White House. I mean, it's not like here where we know we have the laser in the Iron Dome and we're ready for that stuff. I imagine uh that is a concern. How do you >> police it? Who wants to go first? >> I think I think essentially you can't. Um there's really no way to stop someone who has a few hundred drones uh in a city from uh basically pointing them at any target they want. And uh you you can't have every single building with anti- drone laser whatnotss um on it. Uh you can see the cities now in Ukraine have covered themselves in netting. uh every road between cities, all the the major thorough affairs and cities uh have tall poles with netting um so that people can drive without drones uh blowing them up as as they move around the country. >> So is this the future that we want? Um and I want to push back against this idea that well we can't have any arms control agreements >> because someone might cheat. Well, if if that was a valid argument, we would never have any arms control agreements at all. Um, obviously the United States thought that its development of biological weapons in the 1960s would give it a significant technological dominance advantage. Uh, the US was actually trying to develop weapons that would specifically kill people of Slavic descent. Um, and I'm sure the Russians were working on something similar. Um but they decided uh that this was a direction that would lead to the widescale proliferation of weapons of mass destruction that would be very cheap and would be used against the United States. Um so it wasn't revulsion that led to the biological weapons convention. It was common sense. And so uh I think we're in a very similar situation with respect to anti-personnel autonomous weapons. All right, any uh we have to go to break. Anyone have a final thoughts? I just think biological weapons are different than these types of systems because autonomous drones today can be produced using commercial off-the-shelf technologies without the type of expertise or resources that are needed for something like biological weapons.