NorthSec 2025 - Wendy Nather - Keynote: A Tabletop As Big As the World

sharing is I don't know a fishing email. So they would bring a template email to their legal team and say okay this is what we want to share. What parts do we have to redact? Like maybe the name of the executive it was sent to or something like that. But if they got permission for this template ahead of time, then whenever it happened, they could go ahead and send it out. So, these sorts of templates can be really useful to get um to get requests done in advance to your providers. You have even more providers than you realize. Um, another thing that's really good is to have a decision framework to decide who's going to decide ahead of time, who's going to have the ball at any given time. This is a really good resource from Discernible, Inc., which is run by Melanie Ensign and she has great uh advice for people who have to communicate and decide well during an incident. So uh I really recommend looking at that and and looking at her decision framework to see if that's something you want to adopt while you're doing an incident. Um, there are a lot of cases where you may have to buy something really fast during an incident and procurement typically does not take overnight unless you're a very small organization. Um, I can almost guarantee you that you will not have the level of logging that you need for forensics. You won't have it turned on by default. And in fact, I know organizations where they're not even allowed to put an agent on the endpoint until they're pretty sure that they've had a breach and then they can start doing their EDR and stuff. It's a little bit late for that, but in a lot of cases, you have to count on the fact that you're not going to have all of the data that you need right now. You may not have subscribed to the highest tier of logging service. So, you're going to have to get on the phone to your supplier, to your provider, and go, "Can we turn this up really fast because we need this now." So, talk this out during the tabletop. If you need to go out and buy something, if you're not using MFA right now, um I I I worked for Duo Security and we had a lot of incidents at very very big well-known companies where they had to roll out MFA very fast because they had an incident. So plan for fast procurement who's going to do it and who's going to help you. And it's it's good to have a contact at law enforcement that you've already talked to, you already trust, who already knows you. Uh in the US, it can be a problem for example to find the right FBI field office because not all of them can handle a cyber incident. So, you know, it's good to do this homework ahead of time and know anybody else who you might call outside and whether you have permission to call them during a during an incident. So, the the exercise portion of it, exercising this muscle, how many people remember the target breach of many many years ago? Um, you know, it it was the the example of, you know, a really bad breach that they, you know, should have maybe should have been able to deal with better, but they did turn everything around after that. They really came up with a good security program. They even gave tours of their sock. I'd been in there. And they did a lot of things including spending a lot of their time looking at headlines of other things that have happened in other places and going can we detect this in our environment? What would this look like? And if we can detect it, can we automate that detection? So they would go ahead and do that and then that that would free up their analysts to go look at something else that they couldn't automate. So this is another good thing to do during a tabletop. The other thing that they did is they had weekly calls. They had weekly tabletop calls where all the business leaders had to get on the phone whether they wanted to or not. Yes, they would whine about it, but they had they got them used to everybody showing up on this conference call on on the same day and then going, "Okay, here's the scenario. We're not going to need you and you and you. You can you can go back to work. We're going to need you and you and you to do this scenario." So they got used to this muscle and and it was the case where the business leaders in particular all got used to responding to being on the call. And I have to point out, as you probably know, not all the incidents look like the headlines. They don't look like the cool stuff. There's like really really weird um insider things. One time I had to investigate our own inspector general, which is really awkward. We had to wait for him to leave his office at 5:00 p.m. so we could go in and image his machine. It was It's very embarrassing. So things that you're not planning for, you know, practice those sorts of things. Just roll some dice and go, what if it's this person who's an insider? How would we handle that? And uh as I mentioned, it's good to teach other departments and roles how how it looks for them and what we're going to need from them, especially CISADM's uh excuse me, SR engineers. It's a lot it's a lot fancier now than when I was a cisadmin. But anyway, you know, they're going to see stuff way before you are. And it is invaluable to have somebody come in your office and close the door and say, "I think there's something you need to know." Because that's where you're going to get a lot of really good threat intelligence. Uh we've also done some research together with the Sciantia Institute. I did this when I was at Cisco and we actually found that those who did ROSP specific awareness training did a lot better on their incident response than those who hadn't. So, don't just make everybody watch the same, oh, you need to protect data video. Talk to everybody about what it's going to look like for them and what their roles are going to be. Important details like secure communications. Have you figured out yet how you're going to communicate securely in the middle of an incident? Many years ago, I could not even get Microsoft's Thread Intel team to send me a working PGP key. So, you never know until you actually try it out. Um, another time I had to try to get hold of one of the hardest problems is getting hold of somebody who's never heard from you before and trying to convince them that you have something that they need to know. I had to reach out to one retailer and it was about an insider threat who was selling access to their point of sales systems. So, I didn't know where this insider was. I could not send email. I couldn't call their help desk. I couldn't leave a voicemail. I had no idea where they had access to. So, I had to go through LinkedIn and find somebody who knew that CESO. Also, nobody answered their phone because nobody does this anymore. So, it it's kind of tough. The people who do answer the phone, by the way, are executive assistants. If you can find an executive assistant to somebody, they're more likely to answer the phone. But anyway, I went through LinkedIn, found somebody who knew the CISO, and had him go to the CISO and say, "No, really, Wendy Nathther needs to talk to you and and you need to take her call." And I got on the phone with him and said, "Look, I have all this data. I have this threat and tell report that was sent to me. Do you have any fax machines in your building? And he said, "Yeah." And I said, "Okay, go pick one at random and stand in front of it and tell me the number. I'm going to send you this data by fax." Because that was the best thing I could think of at the time. So work out your secure communications in these tabletops ahead of time. And also finally, it's great to do a tabletop because you can help build it and you can be evil. Well, why should the red team have all the fun? You know, you know your business, you know your organization, you know how bad things can get. Let's have some fun and play with it. Um, another thing to do with tabletops is discovery. And really, sometimes people don't know that they have other data centers. I have a friend who who likes to read documentation for fun. I know he's one of those guys. And he was reading third-party contracts for his agency and he found out that there were two acres of additional data center that they didn't know they had. It was in the contract. It was being managed by a third party. I'm sure if things had, you know, gone south, they would have noticed because some large application would have gone down. but it was running fine, so nobody knew. And the address was there in town. So he drove drove over to it and sure enough like I don't know 14,000 15,000 servers and data centers there that nobody knew about. So don't be don't be too surprised. You can discover things like this when you're doing tabletops. Um, yeah, it's a good idea to to figure out whether if you need everybody to change their password at the same time, whether your infrastructure can handle that, especially if you need to do it more than once. Uh, I was also at an organization that had to suddenly get everybody to change their passwords and they couldn't because the infrastructure was melting down. So, that turned into a DOS situation. and what does it look like to operate in a degraded state? A lot of people don't think about that and that is part of resilience. Uh what's the minimum stuff that you can get back up? I was working for a bank when and we were I was based in Chicago when the Chicago River broke through the tunnels and flooded the downtown including the basement which meant you had no more power in the Chicago border of trade building and we had to rebuild our production backbone in two different locations one in New York City and one outside of Chicago. So there were people who were actually carrying down and this is how old this is sun four servers down 11 flights of stairs in the dark. So um what's the minimum that you need to have your operation going back into a kind of operational state until you fix everything else. And then the other assumption of course everybody's just going to come into the office. I have a a friend who gave a great talk. He was working for the Texas Department of Transportation. He wasn't supposed to start his job yet, but they had a ransomware incident. And this was just as the pandemic lockdown started. So, nobody was going into the office. They had just finished locking up all the badge readers. There was one person left in the data center and he had to stay there because he was the firewall administrator and he had to set up VPNs for everybody to come in remotely. So, uh, and the poor guy, he couldn't leave because the door was propped open and and the badge readers had been disabled. So, uh, yeah, it it was a whole it was a whole thing. His talk is really really funny if you ever get a chance to listen to it. uh and identifying gaps. Very important. Who who's really in charge of that? Well, except for this part. I mean, we still have um hang on. I need my vodka. Um there are there are so many things that people, you know, assume or two people are going to try to do the same thing. they're going to step in and try to do the same thing at the same time. So, that's a good time to to do that. Uh, I mentioned logs already. Do you all do you have the logging at the level you need? No, you don't. Does anybody think they have the level of logging that they would need for a forensic investigation? No, it's not turned on. Do you HD? You You kind of do. I'm not surprised. Uh, only HD. Um, and what do we expect from our partners and our customers? How are we going to communicate with them? Do we have to write a tool, for example, for them to run to find out if they're affected by what's going on? That's happened before. And that's why, you know, sometimes you can't notify right away because you you don't want to just say, "Yeah, we don't know if you're affected. Good luck." You want to be able to give them something. And so we had to wait a week or two and say, "Okay, here's the tool that you can run to figure out if you're affected." Uh so practice those things as well. Um, I heard a really great talk by u a a large security vendor who had a breach and they could not notify for six weeks because they did not have per object logging turned on for AWS and uh they had to do this that their stuff was running in in just a little place called US- East1. It took six weeks for AWS to get the logs for them that were just for them at a per object level till they could make sure that they really did have this data accessed and then they could announce. So, these are sorts of things that you have to have good relationships with your providers because no matter what your your um your contract says, you're going to end up asking them for things that you never thought you would. So, and then finally, I mentioned preparing for the unexpected. Uh, does anybody here use chaos engineering at all? Yay. Um, we we also found in the research that we did that anybody who used uh chaos monkeys or the equivalent were uh also much better at incident response because it would break things that they would never have thought would break and it surfaced a lot of things they had to practice. Another cool thing if you do a tabletop is have a key persona key decision maker sit it out. Say, "Okay, you're now on vacation in the Bahamas. Uh you don't have phone signal. We can't get a hold of you. Okay, let's run the exercise without you and see what happens." Uh that there are things like this that you have to do in some banks. Uh when I worked for a Swiss bank, I had to go on vacation for I think it's like two or three weeks and was not allowed any contact so they could surface any fraud. But it's a really good idea to practice having somebody be missing whenever you have to do really fast decisions like during an incident. And of course today there are ripple effects from breaches from providers you have never heard of. Um, has anybody heard of the Blackbod ransomware incident? This was a platform that serviced a lot of nonprofits and uh, again, Scientia did did some research on this and found that the ripple effects from their ransomware incident affected as many as a thousand other organizations and most people had never heard of Blackbod until this happened. So sometimes you will find out that you have second or third tier providers that you don't have the contracts with that you're relying on your provider to manage and suddenly something's happening there and it's hitting you. It doesn't matter how big an organization you have, you can still be affected by a really small organization. Okay, let's get to the fun part. Here are some of the tabletops that I've done. Uh the first one I did was for a Swiss bank and uh we we it was just a few of us in the security team. We sat in a in a conference room and talked through it and the person who was leading the exercise kept asking over and over again, "Have you called the CTO yet as this situation unfolded?" And the guy who was investigating it kept saying, "No, no." And our CTO was saying, "Why haven't you called me yet?" So, I can tell you just about every senior leader wants to hear sooner rather than later about an incident, even if you don't know yet whether it's an incident. Sometimes it can take a while to figure out whether it is, but they don't want to be caught unprepared. So, uh, go and ask your bosses. I'm pretty sure they will all say, if in doubt, tell me sooner rather than later. Even be prepared to say, here's what we know. Here's what we don't know. here's what we need to find out and here's how long we think it's going to take us to find out and I'll let you know again in an hour or in five hours or whatever. You know, I'm I'm calling AWS and they're not calling me back or whatever. Um but you notify them sooner rather than later. The other thing the kind of the flip side is that is be careful of people on your team who are afraid to say anything until they understand everything. I know it's really tough to to have to go and say, "I don't know what's going on." But you got to practice saying that we don't know what's going on, but we think this is what is happening, and here's why we think it. And and you know, explaining that way. Um, and the other thing is that we only had our our top leadership doing this exercise, but it's really better if you run it through first and then you take it back to your own teams and run it with your larger teams because everybody needs to hear the thought process. And it's a good thing to say, "This is what we found out doing it with just the top team, and now we're going to do it this way, and this is what you should know about." And then they're going to surface questions that you hadn't thought of. So, uh, it it's really good to bring it to larger teams. The next size up, the next tabletop I did was run by the US Department of Homeland Security for the retail sector. It was an exercise called Cyberstorm 5. And, you know, it was all just very exciting, flames and and all this sort of stuff. uh and and what they had us do was all call in and they started with a scenario. They started with a set of injects. The funniest thing was they started with um the the first injects were supposed to come from the FBI. Here are the indicators, you know, and this is how the exercise starts. And we all kind of laughed quietly because we knew that the FBI would never be the first to share indicators. But anyway, um it was the scenario was a uh some malware that uh when sent certain commands or when you tried to investigate it would brick the system that it was on. And this was spreading and it was, you know, everything that what we had to do was describe what we were going to do to the runners of the of the the dungeon master to run those who were running the simulation because we couldn't just do anything. There are all sorts of weird things that somebody can do during a tabletop that you're not prepared for no matter how you thought it out. Oh yeah, we're going to take the CEO out and give her three martinis and domain controller access. Do you know what's going to happen if they do that? I don't know. So they kind of herded us through the scenario [snorts] and it turned out when when somebody was finally able to get to get an image of an affected machine, they were able to find a public and private key pair that could be used to uh with the the command and control server to send a disable command to the malware so that they could disable it without it bricking. So you know it was great. The group that found that was the PayPal threat intelligence team. Yay. Good for them. Uh, and at that point, DHS said, "Okay, the exercise is over. We're done." I was like, "What? Wait, there's so much more that you could do with this exercise." First of all, as I mentioned before, the difficulty in doing secure ad hoc communications with somebody, that is a problem no matter where you go. trying to call somebody and saying, "No, really, I'm not trying to sell you anything. I need to talk to your security team." That's really hard to do. I would have liked to see the tabletop exercise include that. And then finally, since they found the public and private key pair, how would you get that key pair covertly to anybody who needed it without the threat actor finding out? because the threat actor was still active and could have sent, you know, brick commands to whatever was still infected. So, could you have stood up a C2 as a service for retailers who didn't know how to do this on their own, but you could set it up so that it could send the disable command for you. If you had been able to set that up, how would you communicate that to the retailers again covertly, securely? That was the challenge that I really wanted the tabletop to take on because I think that's a big problem that we we need to figure out. But anyway, it wasn't my exercise. I got a Secret Service challenge point out of it. It was nice. It was fun. Okay, the next one up, um, I don't know if you've heard of the Atlantic Council, but they do, um, regular cyber 912 strategy challenges that's supposed to be the day after a cyber 911 and what you do about it. And this competition takes place in a lot of different locations around the world. And um it's they have three rounds and uh this is for usually college students, usually grad students, sometimes undergrad teams. If you have never done this, it is a lot of fun. Uh so if you're a college student, try to get together a team and see if you can participate in this. And uh there are international teams and judges. Sometimes they're on-site uh challenges and sometimes they're virtual. uh sometimes teams will will come in on Zoom from Africa or you know Kazakhstan or whatever to take part in these challenges. Um it it's it's enormous. Uh what happens is they give the teams they're usually about 20 25 teams. We give them the first intelligence report with a set of of fictionalized injects and they have three weeks to create a briefing document. Uh, in the US for this competition, they have to pretend that they're briefing the National Security Council. So, they have to come up with a briefing document. They have to come up with a a one-page decision document and a 10-minute presentation. And as judges, we play the NYSE. Now, what's really fun about this is when I'm doing this competition in DC, the judges sitting next to me sometimes are on the National Security Council for real. And so I get to listen to their feedback that they're giving the students. No, that's the wrong agency to handle this. You know, you should have done this, you should have done that. Uh I I learn a lot. After we get to the semifinals, they get this next set of injects and they have overnight to again to write a decision doc and to come up with a 10minute briefing. This is the night when most of these teams do not sleep. They are just up all night figuring out what to do with this inject. Uh this is what one of them looks like. Uh I I did this one. Uh if you've ever seen Joe Men, I got permission from him to to use him as an example. And then finally, intelligence report three, the finalists, they have to read the third set of injects and then they have 15 minutes to prepare their briefing and go in and brief the judges. This is just like real life. They have to read it really, really fast. And then there are three winners at the end of this. So, you know, I get I have fun playing as a judge because I will pretend to be somebody on the NSC or some other policy council and I can either play like I'm a war hawk and I'm ready to to go to war over this or uh I can play stupid and say, "Yeah, I don't understand. Why can't we just do this?" And whatever and see how the students handle it because they're going to be briefing real people. So, they learn how to present their action plans very clearly to these council members, whatever role they're they're playing. Uh, they have to think on their feet because there's a Q&A section and as judges, we ask them about, you know, what does this mean? What is this doing? And and and that sort of thing. And they have to identify everybody who would handle this from a from a policy standpoint, from a government standpoint, what they're going to say to the public, what the president would say, you know, if we had to make an announcement. This is usually not just a nationwide crisis, but an international crisis. What are we going to say to our allies? And I love talking a lot of the the US militarymies take part in this and they're very, you know, oh, you know, we're going to we're going to deny everything. We're going to, you know, do a a an exercise in the Taiwan Strait. We're going to scare these people. And one scenario turned out to be was our bad in the US. The NSA created a backdoor and it was found by a bad actor and exploited. And the NSA said, "No, you can't get rid of it. we still need it. So what do you say to five eyes? What do you say to to other countries who are being affected by this? This is all, you know, an enormous part of the challenge that they have to deal with. So are there laws and precedents for this situation? The the students are usually in law or international relations, policy, cyber security. um they will say things like, "Yeah, we're just going to patch all the cargo ships because they're all hit by ransomware." First of all, if you talk to somebody who actually knows these, you will know that cargo ships reject just about 90% of any patches that you wanted them to try to install. But also, you have them all lining up at docks, it's going to take months to patch all of these cargo ships. I had one team say, "Yeah, we're going to take North Korea off the internet." And I said, "Okay, you're going to do that? How exactly you're going to invade another country and go cut the cables that lead to North Korea? Exactly. How are you going to do this? So, um, you know, they need a lot of help with this sort of thing with with, you know, practical cyber security. So, I got to write a scenario for the competition for Geneva. They asked for it to be in space. So, I got to write a tabletop about a cyber attack on satellites. And you know this is really fun because there's so many political issues like between Switzerland and the EU it's not the same uh you know what are the laws there and everything and also there are the laws of physics to deal with. Has anybody heard of Kesler syndrome? Kesler syndrome is when uh especially in low earth orbit where there are a lot of satellites. If something breaks up the de the debris can hit other things and create more debris and it increases exponentially until suddenly you have debris everywhere hitting everything which is why you can't just go shoot something down anymore. So, um, some of the scen the elements that I wanted the students to deal with are the fact kind of echoing crowd strike that you can't, you know, in in that case you had to walk over and patch things on site. You can't do that in space. You can't walk over and patch it. I had a scenario where the the uh satellites that were taken over were jamming the main communications frequency, so you couldn't stop them. And there are policy gaps between cyber security and space. There are space centers and there are cyber security centers but how do they work together if this is a cyber attack in space? I the scenario I decided to make it activists because you know everybody does nation state actors and it's kind of boring. So these were student activists in Switzerland who were protesting um a billionaire's uh sending of thousands and thousands of satellites into low earth orbit. That that doesn't remind you of anybody, does it? Um so so this this actor in the scenario was American. He was very clueless. He moved to Switzerland and changed his name to something French. Very pretentious of him. Uh he was not cooperative. So what do you do when the private sector will not cooperate with the government? That was a thing I wanted them to explore. And there are more and more deployments. I also had the activists panic because they just wanted to take out some satellites. They didn't realize like they didn't understand Kesler syndrome. They didn't realize how bad this was going to be. And uh also disinformation was making it worse. There were a lot of armchair satellite experts saying why don't you do this and then the policy makers were going in and saying yeah why don't we do that and you know the real scientists were going because that's stupid because physics you can't just do this and so uh it it was it was a great exercise uh I outlined this last January before the inauguration but more and more headlines about about satellites and about them being deorbited and just spraying their uh the the components all over uh and rare elements that they're made of whether that is destroying the ozone and everything [snorts] and these things started popping up after I had written the scenario and I thought oh my god it's coming true um the students thought of things that I hadn't in the very last scenario uh one of the last injects was a little IRC transcript from to the activist going oh my god what have we we have to tell somebody. We didn't think it was going to get this far. So the students, the competing students said, "Why don't even if the billionaire won't talk to us and his company won't talk to us, why don't we go talk to the activists now now that they're scared and you know, maybe they'll help us figure out how to deal with this." And one of the threat actors, it was the the Uganosa in in Zurich. They have a space program. So I made them the source of some of the activists and their team from the atsuru actually won. So I thought that was really cool. I was not there to hear when they when they read the inject and said wait a minute that's us. So why does it matter all to us wrapping up here because even if it's a local tabletop exercise it's the the world's going to come crashing in. We are too intertwined now as an ecosystem for it to be just one organization that has an incident. I I have not heard of any incidents anymore that do not affect, you know, don't have ripple effect somewhere. Um and even though college students are smarter than I was at the time, uh a lot of them still don't have that real world nah that's never going to happen sort of experience with cyber security. So also when you're doing these, try to talk through these scenarios outside of your areas of expertise. Don't just focus on what you know. Go ahead and explore because you know it's just a tabletop. So thank you to everybody and I will look forward to seeing you here at the conference. [music] >> [music]