Boiling A Frog: Science & Ethics as TTRPG - EMF 2026

[laughter and gasps] So, my name is Mariam. Um, my background is actually in astrophysics. It made a lot of sense to come this year with the space theme, but then I ended up giving a talk about ethics instead of anything space related. So my current position is at the University of Cambridge as a member of the ethics, science and the public project. I am in academia which means my job is kind of fake. As long as I can get the money to do something, I can do it. Which means given I convinced some funders that it would be a really good idea to develop a TTRPG on the premise of science ethics. That is what I'm currently doing. Um so that's what I want to talk to you about. But given how how um the talk slots here are only 30 minutes, we can't actually play anything. So instead, I am going to introduce you a little bit to my work, the background, and then hopefully we'll have time to have a little some some discussions amongst people. Um I won't make you feedback to the to the class or anything like that. Don't worry. [laughter] So to begin with, this talk is called boiling a frog. So, for those of you who don't know, there is an apocryphal story about how when a frog is placed in boiling water, it will jump out. But if it's placed in tepid water that is slowly heated, it doesn't recognize the danger and it dies. Now, this is a metaphor. The story itself isn't true. Since frogs do jump out of water if it's boiling [laughter] slowly, they're not that stupid. Um, and also, if you drop a frog in boiling water, it won't jump out because it will immediately die. So this story is [laughter] is not actually true. Um but the metaphor is still important. It's about the fact that we live in a world where we've seen this happening. We've seen how people are unwilling to recognize danger that creeps up on us slowly while we're comfortable. So the discovery of nuclear fishision and a no Nobel Prize, which I think you can all tell where I'm going with this. It went on to become the foundation of nuclear weaponry. Um, it's also not just a thing of the past. AI is the easiest example that has started off as an annoyance to some of us and a convenience to others. But now we do have the possibility of autonomous weaponry and humanitarian disasters comparable to the nuclear bomb. Quantum computing is also in our future. Um when I was doing my undergraduate degree, the professor who took us for our quantum class told us that cracking that secret to quantum computing would mean we had the power to decrypt the um protections around nuclear codes. So it is just a very active race between countries right now which terrified me and I did not then study quantum any further. [laughter] So naturally, I think the question is why are we boiling the water? Like what's the point? Why not just why not just not do that? Um so in my time working at the intersection of ethics and science, I have come to appreciate a few things. Firstly, ethics can be quite a tickbox exercise for a lot of cutting edge researchers. And I don't mean that in a disparaging way. I mean it in that scientists can't always see the forest for the trees. It's a really human problem, I think, to be so focused on progress that you forget to sort of look up and notice what direction you're progressing in. There's also the very important question of funding. So I have a friend um who's doing satellite research up in Durham and their group is aiming to uh create global equality in in terms of assisting with digital poverty in the global south that kind of thing. But the projects are of course funded by the defense sector and that satellite development is not only going to be used for those very noble goals. Um, defense is actually one of the biggest funders of science and funding in science for any of you who are in in the field will probably know is not easy to get. Research is not an easy place to be in terms of the funding landscape. So people do just kind of have to do the work and research that is funded. [snorts] So is there anything we can do about it? So, some of these problems are bigger than researchers themselves. They're even bigger than the funders. I think I'm the first to be anxious about defense funding. Like I was just saying when I was at um when I was doing my undergraduate, BA BAE systems were like knocking the door down like all the time. They were fliring us constantly to try and get us to to work for them after our degrees. And I turned them down immediately. Of course I did. But I look at Trump and I look at what's happening in America and a part of me does want us to be able to defend ourselves if something like that came for us. And it's really complicated. I think it is it isn't quite so straightforward as simply don't. I don't think um but that isn't true for everything. I don't think everything is beholden to those kinds of uh stringents and and rules. There is um a precedent singular I think sadly but there is a precedent for ethical self-regulation within science. Does anyone here know about the Oyama conference? Does this sound familiar to anyone? We have one hand two hands. Incredible. [laughter] Um, so I could actually spend the entire hour talking about this, but I'll I'll try and keep it brief because I do want to give you a a good amount of time to to have some meaningful conversations amongst yourselves. Um, so in 1975, the field of genetics became a really really good example about how the scientific community can self-regulate in the face of existential risk. Basically um rapid advancements were being made in the field of genetic science and noble laureates including people like Watson of of Watson Cric and Fankran um in their DNA discovery team called for a moratorum. So a global moratorum was put in place. Nobody was allowed to continue their research. And when I say allowed, it functioned almost entirely on reputation because science as a field is very heavily reputational driven. This worked when the people who are in charge of journals and everything that can dictate your career turn around and say you're not going to research that. You listen. There were of course a few exceptions to this in countries where the um leading experts were not supporting the moratorium but it was a very small minority. The majority of those researchers went okay we're going to just take a pause for a second. They put together a conference in a cyar which is why it's referred to in this way and they came together over three days and just discussed heatedly what the future of the field should look like what it is ethical to do what it is not ethical to do. By the end of those days, they had not reached a consensus, but they had reached enough of one that they could sit down and put together essentially a regulatory document that determined what you could do in your lab, what you could not do in your lab, and what you could do in your lab given you had a ton of safeguards in place. There's a Rolling Stones article about it, which is brilliant, and that I recommend you all read. It was actually really interesting that the Rolling Stone journalists were allowed into the conference itself at all because at first the scientists denied them. They said they weren't allowed to come in and they turned the journalists turned around and said, "Okay, we're going to report on it either way. If we're not there, the headline is going to be something like scientists are holding a secret secret discussion about DNA modification and then they let them in." So, [laughter] so it was very very good on their part. I think I do work a lot in public communication as well and I have a lot of respect for that. Um, they also invited lawyers because a lot of what they were talking about regulation crossed over with policy and with the legality of what they were trying to do. But when do we make that call? So that moratorium was called in time. When does the frog start to die? So there was a fairly easy turning point. I say easy, there was a fairly terrifying turning point for the Ayamar conference in that a guy called Paul Berg decided to focus on something called the SV40 virus as a potential carrier for introducing new genes into animal cells. So he was going to use E.coli in in these experiments which is really common. um it's a very common strain of of bacterium that is often used in in such laboratories. Now SP40 was known to cause cancer tumors in mice and I think the idea was if you can work out how this is working you can then develop a a cure etc. I think that was the idea. My background is physics biology is one of the only sciences I didn't even try so please don't quiz me on that. Um, but E.coli is of course a very common bacteria that lives in the human gut. So Berg's planned experiment raised the possibility of creating a highly transmissible virally caused cancer that lived in a bacterium which everyone carries in their gut. If that was escaped from the laboratory conditions, you would have had essentially an epidemic of cancer. like it would have been really bad. So that proposed research um led to the community realizing that a really harmful really really harmful um gene could just be introduced into a new organism and then into into humankind and was a key factor in the moratorum. It was a very obvious red flag. Perhaps controversially, I do think AI has already had that has already had very obvious red flags and has already had moments where we probably should have called a moratorum. I think if nothing else, the backlash from things like art theft and those legal um battles should have instigated something like that. But I think the difference is the scientific community was as I said before completely um at the whimsy of of of like their representation. that was the thing that was most important to them. That was the thing that drove their careers. Whereas a lot of the things that drive AI are more commercial and are capitalist. So the same the same um I suppose regulatory drive isn't there because as long as it's making money it doesn't really matter. But also with the genetic example they didn't stop researching. Research into genetic technology has led us into some really incredible health discoveries. Now, they do come with their own ethical considerations. For example, the fact that we can edit genes to cure illness leads to a conversation about what classes as a curable illness. There is I did a lot of work with the deaf community last year who consider deafness a part of their culture and heritage. And so when deafness was included in the list of genes that could be edited out, they were not happy. The actual the sign language for um for for genetics is the same as for eugenics because they were that horrified by the prospect of being edited out of society, which is very valid, I think. Um, so it does the the continuation of that research does lead to its own ethical considerations and can and things that we need to to raise and have discussions about. It's why science communication and public engagement is so important also underfunded. Um, so the argument is if we were too scared to push the boundaries at all, we'd still be seeing plague doctors using leeches, etc. Yes, there are things to consider in terms of what we do and don't cure, etc. But the existence of that technology isn't inherently bad and the reason we have it is because we continued to research. Yes, we called the moratorium. We worked out what we should and shouldn't do, but we then continued. So, how early is early enough to try and work out if you continue, if you need to fork the path? How early is too early? As in you impede progress. You you stop developments into research that could change the lives of hundreds and thousands of people for the better. We also did a uh citizens jury in my center where the question was put to I think it was 45 45 people with lifelimiting single gene conditions. The idea was asking them if in the future and it was a very like very loosely not loosely but with a ton of uncertainty and in in it in the phrasing of the question if there was the possibility in the future of researching the cure for single uh gene genetic illnesses because that's all we're talking about here right now with the technology. We can only um deal with single gene illnesses. um would you want that research to take place? And this was a room full of people with incredibly emotionally charged stories to tell about how they really don't want to pass on the illnesses they have to their children. They don't want to see them suffering. Um versus people who are sitting there going, "But where does where's the line? If we say yes to that, when does it become worse?" Um, and ultimately though they voted in favor of that research happening, it was not a unanimous vote and notably it was the people of color in the room who voted against who just didn't have the trust in the science. Um, which I think given the historical injustices in health research makes sense but it's also really sad and really interesting. Um, so this isn't just a question we can just answer. We can't just say, "Ah, when you make a 60% progress, stop and think about ethics." That's not how this works. It's so much trickier than that. So much more complicated. [snorts] It's something we have to think about constantly as researchers. It's something that at every stage you have to consider. But when there isn't an application immediately visible, it's hard. It's hard to conceptualize whether or not this the thing you are doing in complete isolation is going to be bad for society 10 years down the line. It's difficult. I think this is why the AI researchers have had such trouble because they're mathematicians. No mathematician is sitting doing a equation on a whiteboard thinking, "Ah, yes, I should I should definitely consider the ethics of this equation." They're all mathematicians. They've never had any any background in ethics. Biology is probably the only field that has really good ethical um discourse and and education available to its undergraduates. I didn't have any ethics training on my undergraduate degree either. And when I came into this job, the first things in my mind about ethics of science were things like Monarchya. I don't know if you guys are aware, but there was a telescope being built in Mona which was on sacred ground. Um so the local community opposed it. The global community and the the local government had approved it because astronomy is a global endeavor. We only have one planet to try and view the entire cosmos etc etc. So, it's really important to use the one planet we have as effectively as we can to try and get that data. But the people of Monarchia didn't care about that. They cared about their volcano. And that was what came to my mind, these kinds of cultural implications because I've never had any ethics training. And I didn't realize that things like reccominant DNA were real. So, [laughter] um, so it's really hard is what I'm getting at to to not only to give researchers the training to have them think about it in a way that isn't going to make them feel like their hands are being tied and they need to tick these boxes before they're allowed to do their very important research. How do we train ourselves to have the inherent mindset to really think about where this could go without being bogged too bogged down and too anxious about the consequences to make any progress at all? Slight tone shift. I think the answer [laughter] to those questions is actually creativity. I think we need to meet researchers and scientists where they are and help them to develop more open mindsets through the through the medium of creativity. And to that end, I am developing the science ethics tabletop roleplay game that I mentioned at the start of this talk. So, I did assume everyone here would know what tabletop roleplay was, but our good friend Tim here opened this talk by telling me he thought it was a did you say a train? >> Motorcycle. a motorcycle race. Um, so I will give a very brief overview. Tape top roleplay are of course games like Dungeons and Dragons where you have a game master who guides you through a story that you, the players, have a huge amount of control over. So there are obviously usually things you are supposed to do to the to the dismay of many a DM. People often don't do them. But there are things you are supposed to do and ways you are able to get there. And those decisions are all your own. And campaign modules of course exist for these games and I'm developing one of those. So empirically I think scientists exist in board game communities. I think scientists and researchers and people who are into like nerdy crap. Pardon my French. It's it's a circle. That's not a ven diagram. That's a circle. [laughter] So I think that there is a lot of potential in in meeting researchers where they are in that way. The same way we meet publics where they are. We are really careful in in public engagement and community engagement work not to expect communities and publics to come to us and to suddenly be interested in science because why would they? They have no reason to care. It's your responsibility to meet them where they are and bring them into into the conversations that you want them to be in. And in the same way, I think walking into a lab and being like pew, I'm the ethics police, it's not going to it's not going to work for researchers and for scientists. I think they need a space that is more aware of who they are as people and of what they need and what they enjoy. And the reason I actually came to this conclusion was because of the TV show Arcane. Have has any I'm going to just Who's watched Arcane? Yeah, I figured there'd be a decent number of people who had watched Arcane in this crowd. Um, so I watched that show and I only watched it because my friend and I do a lot of cosplay and and she came up to me and was like, "Please, I want to cosplay Jason Victor." So, I was like, "All right, I'm going to watch it." Um, I got to to season two and enjoyed it. Season one felt didn't didn't land with me, but then on my rewatch, I realized that this is show like magic science ethics storyline over two seasons that people were watching because they wanted to. People were engaging with the ethics of science without realizing it because they wanted to. And that was like magic to me. Um, and then that first season there was a moment where the man who turns out to be the villain says something like, "When you're going to change the world, don't ask for permission." And I watched that thinking, "No, you do ask for permission. What an entitled white man thing to say." Like, "Yes, you ask for permission." And then it turned out that he was the villain. And I was like, "Oh, okay, good." Um, and that that sort of like swap from a Yeah. to boldly go etc. Being being bold and brave in science is the most important thing. But then refraraming that as actually quite an unethical and negative thing to do really really resonated with me. I thought it was really good and obviously this is a League of Legends TV show and I was like I know who plays League of Legends. So I was like this is a medium for engaging exactly the people who I want to engage with ethics with ethics and that made me think about TTRPG and those kinds of places where researchers and scientists live anyway. So instead of movies and TV shows which have been creative methods of engagement in the past, I think a TTRPG encourages more participation, more thought, more active engagement than something like a TV show does. The reason I also prefer the sort of subtlety of Arcane over things like Jurassic Park is in Jurassic Park, you kind of knew it was a bad idea. This guy's bringing dinosaurs back and you're like, why? [laughter] Whereas in Arcane, everything was framed as a as a benefit to society. there were so many very obvious positive potential results for the entire city for the work that they were doing and then it just turns out to not kind of go that way which is a bit more realistic I think. Um, so I think bringing people into stories in that way is really powerful and and really shows what we're trying to show in terms of building that mindset that where could it go attitude. So using creativity as a facilitation tool and the idea of anonymity and theater almost as a protective blanket is the direction I'm going in with this. I think hopefully it means researchers and scientists aren't going to be on the back foot, aren't going to be worrying about losing funding if their ethics isn't right because that's part of it, right? You have to tick the boxes and you have to be like, "Yes, I'm so ethical." Because otherwise you don't get funded and that's your livelihood. You can't actively really honestly think about the ethical implications if you need the money. I think it also gives people the freedom to step into someone else's shoes and play through a fantasy version of these problems that is completely removed from their own lives to then develop the skills required to think ethically without that pressure. I have a game designer picking a system and developing the adventure module right now. So I can't actually give you the full rundown of the different roots in the story, but I would like to give you a bit of a flavor. So I have a biology route, a physics route, and a chemistry in production. Um, and of course the aim here wasn't was to make sure we weren't only focusing on that is the frog boiling yet moment like a like a are we there yet? Are we there yet? We didn't want that. We wanted to create something more holistic that helps researchers and scientists think more broadly about all of the ethics of their work, including things like public interest and socioeconomic factors. I think the key to socially responsible science is getting out of this the pursuit of science itself is so perfect that we don't need to bother mindset. The roots as I said are still in development and actually I am looking for both researchers to help develop the the chemistry and physics roots as well as players for focus groups. So if any of this looks interesting to you, there's a there's a QR code at the end that please stand for that. Thank you very much. Um so our first route our biology route is about a magic related plague that has been ravaging a country. So both traditional medicine and magical cures have proven ineffective and the death count is continuing to steadily climb. You learn about the city, its inhabitants, the different problems they're facing and about a miracle cure that has appeared as of late. In our physics route, we're looking at a breakthrough in energy generation, but where do the gains outweigh the cost? This is about secrets that both you and your collaborators are keeping as you explore the benefits and consequences of this new technology. And then finally, the chemistry route is about a new alchemical process that has reduced cost and sped up hundreds of tasks in the daily lives of the citizenry. but they're unable to see the ecological cost and the public is just really happy about this. So this is almost about working against public opinion wherein the researchers would be the ones bringing the public's hopes down instead of having to big up their research in the face of public dismay. So I only have 5 minutes left but I would like to do a little bit of of thinking in in in groups. So, what I have is a couple of um well, I had two scenarios, but I might skip to the last one, which takes you through some of the motives, potential motives, shall we say, in the biology route of the game, which is why this presentation has all been like biology themed TM. Um, and what I want you to do is just talk to each other a little bit about what I'm about to put on screen and about what it makes you think of in terms of what problems are there in this situation. You don't need any answers. I don't expect you to solve ethics in 5 minutes. It's not what I'm asking for. Um, I'd just like to to, you know, give you some prompts to to think about a little bit. I will read both slides, but instead of giving you a gap in the middle, I will just let you talk about both slides at once. Um, so the first slide, a young woman who is suffering suffering horrifically from that magic plague has been miraculously cured. She's been locked away in an old decrepit house for weeks because the rest of the village has been incredibly fearful of her. She couldn't afford any of the experimental treatments on the market and so turned to this mysterious traveler who claimed to be able to cure her for free. His cure worked and talk of his cure is now spreading like wildfire, but the man himself has disappeared. So, actually, I will just leave that one up. Would you like to make friends, everybody? [laughter] Terrifying, I know. I'll give you a couple of minutes just to talk about that. Hello. I'll be your friend. Hello. Am I back on? I'm back on. Okay. Hello everybody. I have returned. I'm sorry that was very short, but it's Yeah, I only have a minute left. So, um I would have liked to do one more of those and also ask you if there was anyone who was wanting to share. Um, but I only have a minute left and I do really want to put my QR code up. I'm not gonna lie. So, there's my QR code. [laughter] Thank you so much everyone for coming. I hope this has been interesting and enjoyable for you. Um, yeah. Thanks for listening to me ramble about things that make me mad in science basically. >> Oh, [laughter] I'll fix the QR code. Please find me in the Q&A tent if you'd be interested. Thank you. >> [applause] >> Thank you very much, Dr. Mariam. >> I will um I'll fix it and share the in the Q&A tent. Thank you. [laughter] >> So, I believe Dr. Mariam is going to be going over to the QA tent for a little while if anybody wishes to uh talk to her, ask any questions. Um and yeah. Oh, thank you very much. Can we have a round of applause, please? [applause]