Three More AI Hacking Incidents, and a Push to 'Pace the Frontier'

[music] Welcome back to the AI policy podcast. I'm Oluk Metha, director of the Wadwani AI Center here at the Center for Strategic and International Studies. >> And I'm Nicole Herrera, a researcher with the center. Today we're following up on our last news roundup coverage of the OpenAI hugging face cyber incident in which several OpenAI models escaped containment and broke into HuggingFac's production infrastructure during internal cyber evaluations. We've got a lot to talk about on that front, including the revelation that OpenAI's competitor, Anthropic, experienced three similar incidents with its own claude models, plus a petition from over 1,300 Frontier Lab employees calling for the US government to help pace the Frontier and an open letter from Nvidia on the merits of openweight models. But Aloque, before we dive into all that, there are two smaller stories I really want to make sure we have the chance to discuss today. All right, great. Let's get started. >> Great. And the the first story is that on August 3rd, Texas Governor Greg Abbott issued a directive to the Public Utility Commission and Electric Reliability Council of Texas to conduct a comprehensive verification and audit of all data centers currently seeking to connect to the state's electric grid. Under this new order, each data center developer will have to provide information about any financial assistance it expects to receive from the state, the data center's projected power and water consumption, measures the developer plans to take to reduce impact on the local community, and who the controlling interests are in the project. Abbott said that the Electric Reliability Council is currently processing around 474 gawatts of requests to connect to the Texas grid, which is more than five times the state's the state's record peak electricity demand, and that approximately 90% of those new power requests are for data centers. Any data center developer that fails to go through this new auditing process will be denied connection to the grid. But some critics are saying that Abbott's directive doesn't go far enough. Uh for example, Texas Department of Agriculture Commissioner Sid Miller said that without legislative action, this directive is all hat and no cattle, empty political rhetoric wrapped in meaningless fluff. So Aloque, what's your take on this new verification and audit requirement? Is this just a case of checkbox compliance or will it translate into meaningful accountability for developers looking to build data centers in Texas? Yeah, so this is a really interesting development. I think before we get to the sort of specifics of what's happening in Texas, I think the one of the interesting things to note here is sort of what it signals about the larger data center conversation happening in this country. So you know just a couple of weeks ago um on this podcast we talked about the fact that New York uh state has put a moratorium on data centers. So that is a very blue state. Here we have Texas a very red state um historically known as one of the um most businessfriendly states in the country perhaps the most businessfriendly um doing something similar. So this is not exactly the same as what New York is doing but it has a lot of similar elements to it. And so I think this shows the extent to which this data center backlash, we've talked about it numerous times on this podcast and in sort of our uh various public events and reports um that this extends across political divides. It's a red state and a blue state thing and it shows the depth to which the American public is sort of concerned about data centers going up. Um I think it also shows uh you know increasing concerns by governors about how they can leverage all this money that's going into the data center industry. How they can leverage that to improve the infrastructure that is um being built in their states. Um including a backlog of sort of existing deferred maintenance and things like that relating to water and power um utilities. uh as well as sort of minimizing public impacts, community impacts because that has a direct connection to sort of how they think about their constituents and how they think about their own political prospects. Um that being said, I think uh here there is probably um you know two things happening. One is that the the the fact that the governor is taking this action is a pretty strong signal about um the level of concern that has risen up to the top levels of Texas government about data centers and that this sort of demand for electricity that is significantly exceeds the state's capacity is a real issue. Um but also that there's probably a limited amount of things that the executive branch in Texas can do without legislative support. And so I think that that there's some signaling happening here. Um and that the most likely thing that will happen is that this signaling will um both change how data center developers think about sort of building in Texas and whether Texas is the right place to build or if they should think about other locations, but also likely to spur some more intensive legislative action on the part of the of the uh Texas lawmakers about sort of coming up with a comprehensive legislative approach with teeth to the data center issue and making sure that they're minimizing impact. >> Right? That makes a lot of sense. And the second story that I wanted to talk about today is a Bloomberg article published on August 4th, the day we're recording this episode. And that article was titled China's AI Blitz Creates Death Zone for rival US model makers. And this title is referring to a chart produced by artificial analysis, an independent AI benchmarking and analysis firm that plots a bunch of AI models on two axes. So on your x-axis you have cost per tax and on the y-axis you have the model's intelligence index. And I realize this chart might be a little difficult to visualize for those for listeners who aren't watching on YouTube. So I'll try and keep it relatively simple. Essentially, if you're an AI user, your ideal quadrant on this chart is going to be in the top left. So, where you have very high model capability on the y-axis at a very low cost on the x-axis. You're going to avoid using models that fall into the bottom right quadrant where you have a lower capability on the y-axis at a higher cost on the x-axis. And this bottom right quadrant is what some experts are calling the death zone for Frontier AI developers. Because if your newest model is more expensive and less capable than say a openweight model from a Chinese lab, you're going to lose customers very quickly. And as openweight models become cheaper and more capable, that death zone is going to just keep expanding. So Aloque, what can you tell us about this chart and the broader imp implications for AI policy, especially as and we'll talk about this more as part of our main topic for this episode, but as there's a growing camp of AI experts and industry leaders who see openweight models as a critical input for AI safety. >> Yeah. So I think uh a couple of interesting things to note here. So, you know, traditionally when we hear people discuss the efficiency of AI models, they're often um they they often talk about sort of token efficiency. Um, and in this in this case, we're talking about cost per task. So, I think that is getting closer to what people care about in the real world is not uh how much a particular token cost because a token is kind of an abstract AI thing that doesn't really correlate with uh real things, real artifacts in in the real world. um what people care about is how much money will it cost me to get this thing done. And so I think that this is closer to measuring the things we we really care about when we talk about how efficient um our particular models. Um in terms of the argument here, I mean death zone I think is is a little bit dramatic. What we what we know is that um measuring AI capabilities are really complicated. Um there is a real sense in which you know some of our benchmarks are saturated so they don't tell us good information about what models are better than others um and that uh in other cases you can take steps to sort of optimize how your model performs in a benchmark but it doesn't necessarily reflect in the real world um that I think there is some truth to this idea that uh Chinese competition will make certain kinds of uh model products difficult um and it'll change the economics of those products and yes certainly if you are offering a less capable model at a higher price that is generally going to be a difficult market segment to be on um but there are I think there are instances where that might that product might make sense right so if this is a product that has associated infrastructure around it if the tooling is really good if the interface is really good if you have sort of a bunch of information associated with that company and so you can port that memory over. It might be useful. And then there are things like cyber security guarantees and cyber security practices that might mean that you are willing to pay more money um for a less capable model because of um things like the the level of security practices, the ability to handle classified information, the level of uptime. Um and so so I think this is complicated. I think it's directionally uh telling us something important. Um but but we'll have to see um how it plays out. I think the other thing to keep in mind is we we don't know or we don't we don't really fully know um how much Chinese models are being subsidized. Um you know, US models are being subsidized by venture capital money as well. And so it's a little bit difficult to get at the true cost per token. And we also don't know uh in the future will China's lack of compute really change the calculus for the kinds of models it's able to serve. Um so we'll have to see. We know that export controls are affecting their ability to um get access to compute and we we'll see if that changes their ability to do inference right to serve models for customers in the future in a significant way. >> Yeah absolutely and thank you for unpacking that. Um, but now I want to move on to our main topic for today's episode, which is that over the past few weeks, there's been a ton of stories centered around AI and cyber security. And this was kind of kicked off by OpenAI's disclosure, which I mentioned briefly at the top of this episode, in which you and Matt discussed in greater detail in the last news roundup, that several of its models had broken out of their internal testing environment, gained access to the internet, and hacked into HuggingFac's production infrastructure. Now, the second big revelation came 9 days after that initial announcement from OpenAI when Anthropic shared that it had also discovered three of its own incidents in which a claude model hacked into another company during internal cyber evaluations. In a blog post on July 30th titled Investigating three realworld incidents in our cyber security evaluations, Anthropic said that following OpenAI's disclosure, it had begun a largecale retrospective review of its own cyber security evaluations in which it looked for evidence that Claude, like the open AI models that accessed hugging face, was able to access the internet from from within testing environments that should have been sealed off. So, Aloque, walk us through what happened here. What exactly did Anthropic discover in its review and how do these three incidents compare to what happened with OpenAI? >> Yeah, so uh obviously um you know, we talked a lot last episode about what it means for a model to escape containment like this. Uh the fact that it's happened at other companies suggests this is a more widespread issue. Um and raises the alarm level, which I think was already pretty high. uh considering just how sci-fi feeling that previous incident was. I think in the the anthropic case we have uh two uh pretty significant differences. So one is in how these incidents were discovered and then the other is uh related to how they happened. So for Anthropic, this was um done because they read about the OpenAI incident and they decided to go back and review their logs and so they found it through this retrospective review which suggests that um you know something they had this data available and they they found it by by scanning their data. Um but it suggests that their processes for detecting these in real time are not working at least the way we would hope in the AI industry. Um, so you know, I think it's really good that they found these, that they're engaged in public transparency about the process of finding them and what happened in these instances. Um, but I do think it suggests that we probably have improvements to make in the continuous detection and evaluation process for testing models. I think the second uh interesting thing here is that um you know there's a pretty significant difference in kind between what anthropic discovered happened and what uh OpenAI discovered. So in Anthropic's case it was uh essentially a seems like a configuration mistake where Anthropic is working with a third party uh a third party evaluation firm who was setting up an environment and there was a misconfiguration that allowed the models to to reach the internet. Um, in a lot of ways the open incident is more concerning because it involved the model uh essentially after being given its instructions detecting a novel vulnerability that allowed it to circumvent the sandbox environment and gain access to the internet. So there was a much more sort of directed action happening in this particular circumstance. Um, in both cases I think what we've seen is that there was um a fairly open-ended prompt. Um, so basically for anthropic, it was, you know, like a a capture the flag challenge. Um, or sort of a scenario set up. It had to find some piece of uh secret information. Um, and then sort of a pretty open-ended prompt and a a pretty open-ended um, ability for the model to sort of do what it needed to do to sort of uh, find this particular flag or or piece of information um, with not a lot of specificity on methods or means or anything like that. And I think this was similar to the OpenAI incident and that there was also an open-ended prompt. I mean it seems like the fact that if you uh don't provide a lot of specificity to the model that there's a greater likelihood of this kind of behavior happening. I think this suggests that um like we mentioned last time there may be some utility in thinking about uh how we do our uh sort of evaluations uh whether we should use open-ended prompts like this and if we do I think there's useful information you can get about capabilities whether there should be additional safeguards put in place when you're using open-ended prompting. >> Yeah. And even before this story broke, we were actually planning on returning to that original OpenAI hugging face incident in today's episode because new details about that incident have emerged since you and Matt covered it two weeks ago. So, can you give us a quick update on that front? What new information have we learned about OpenAI's model escaping containment? >> So, you know, there is some more technical information about what happened. This includes sort of a write up from Hugging Face. Um, and we're hoping uh here at the center to to do a little bit more of a deep dive and and talk through sort of the technical details. So, um, stay tuned for that. We're we're hoping to to release some findings or maybe do some public events on that in the future. Um, but in the meantime, um, I think there there are some interesting uh sort of both industry and policy developments coming out of this incident. So the first is that there is now uh an independent review planned um for this incident. So so Meter which is one of the leading independent frontier capabilities firms um has reached an agreement with OpenAI to conduct an independent review of this incident. Um uh they're going to be working with Redwood Research, uh well-known firm in the space. Um and that the plan is for them to publish um sort of a blog post that lays out some of their uh the details of how they engage with this, what they did, the scope of the investigation, and some of their conclusions. Um and so we we have some details about how they're going to go about doing this work. Um, but I I think we should commend the companies for the level of transparency that they've both disclosed in terms of what happened and their uh willingness to engage with independent um evaluators for uh more more deep dives into what happened. This is very much in line with some of the things we've been talking about on this podcast in terms of the utility of independent verification and what that signals to the public in terms of how much they should trust the model, but also um the fact that they can often provide unique kinds of information and unique kinds of expertise that might not u be that might not reside in the labs. Um and so I think that there is a lot of utility in this approach as well. Um we have also seen some policy responses uh to this incident. So this includes um interest from state attorney generals. So um you know uh just I think yesterday actually about 15 state attorneys sent a letter to uh Open AI um and to the CEO Sam Alman um telling him to preserve relevant documents um and I think um maybe more interestingly halt certain internal cyber evaluations. Um, the letter flags concerns about potential violations of state and federal law, including consumer protection and data privacy statutes. Um, which is something that attorney generals and states are are charged with enforcing. Sometimes they do this alone. Sometimes they do this with the federal government. Um, and so, uh, you know, I think, um, I think some of their asks are in line with things that open said it would do, particularly around the the halting of evaluations, but I think that it is, uh, quite interesting that the state attorney g I mean, it's not interesting that state attorney generals are um, taking interest in this. I think the fact that they are using some of the existing laws on the books um is totally in line with with stuff we've talked about before which is that you know time and time again we've seen that there are lots of existing laws that can apply to AI in some way and that when there is an incident like this there is a lot of sort of uh creative use um or or sort of uh people looking deeper at the existing statutes and figuring out how they can take action. So I think this is another uh instance of the of of people looking at the existing laws and saying hey there is there are existing statutes that cover this kind of activity and we're going to explore the authorities we have under those statutes to protect consumers and prevent this kind of thing from happening again. That's at the state level. We have seen um interest at the federal level as well. So we have some representatives also writing to OpenAI um requesting more information about this um including raising questions about uh the environment in which OpenAI is securing its models and testing its models and um questions about how uh models like this were able to escape those containment processes. >> Yeah. And something that I found pretty troubling when I was reading um some of the reporting on these incidents is that it sounds like these uh weren't the only this wasn't the only time that something like this has happened at OpenAI. Um one time article uh cited an OpenAI staffer saying that externally this this being the the hugging face incident feels like a big warning shot, but internally related incidents have been happening for a while. Um, so that's something that's pretty concerning. And we're also seeing a level of concern from inside those Frontier Labs. Um, there's been a petition now signed by over 1,300 employees of Leading Frontier Labs in the US calling for the federal government to help develop mechanisms to pace the frontier of AI development. So, can you talk a little bit about that petition um, and what it kind of tells us about the the current state of the AI industry? Yeah, it's a really interesting um you know, 1300 employees, Frontier Labs. Um there the the specific ask here is for the US government to help figure out a way to pace the frontier of AI development. Um so, uh noticeably, right, we request the US government to support an international effort to develop technical and governance tools needed to deliberately pace the frontier of automated AI development. Um we've seen a lot of notable sort of figures in the AI industry sign this. This included uh the anthropic CEO Daro Ammedday. Um you know high ranking officials from open AI and meta and Google deep mind. Um so uh that includes the open AI chief scientist Jacob Pachowski uh Meta's AI chief scientist um Google DeepMind's co-founder Shane Le uh OpenAI's head of strategic futures Dean Ball who who recently joined OpenAI. Um Sam Alman didn't sign it but he but he has mentioned similar concepts in some of his public statements including a podcast on July 31st. I think the the the challenge here, right, is that I think the oftentimes what I describe here is we're in this kind of um prisoners dilemma. So even if there's broad agreement that you know developments in AI are happening too quickly and then maybe like in the cyber security space there this is like the most concrete manifestation of that happening too quickly. If any individual company sort of decides to stop development on their own, they'll essentially be putting themselves in very precarious financial circumstances. Um pos possibly or or probably bankrupt bankrupting themselves. Um and so it's very difficult to envision a mechanism where one company is able to do this unilaterally. So you have to have a lot of companies do it at once. But then you have a a second layer to this which is let's say that we can get all the US companies to to sort of sign up to this then we have international companies who are also developing this and then the same sort of circumstances apply which is that uh if the US does this unilaterally and China doesn't then China will continue to make advant advantages take more market share and sort of uh the US will sort of uh drive itself into possible irrelevancy. And so the the real challenge here is essentially the challenge we've had in the AI space for a long time, which is that we can't pace the development of AI until we figure out an international approach to AI. That includes uh the US, China, and a bunch of other countries where there's progress happening um near or at the frontier of AI. And that's a really uh difficult thing to envision happening. Um perhaps you know we have an upcoming summit where the US and China are going to be talking. AI is supposedly on the agenda and I think that it would make sense given all the recent developments we've seen in the cyber space um to think about um that as an opportunity to have some discussions about whether we can come to some sort of agreement, some governance agreement that would allow us to u to start, you know, taking steps towards uh slowing things down. But I think it's really hard. Um, I think that there's very likely lots of other things that the US and China will want to talk about at that summit. And so we'll just have to see what happens. Yeah. And and something else that's interesting about this um petition, something I've seen pointed out by uh Neil Chilson specifically on X of the Abundance Institute is that um and I I'm quoting here from from Neil, uh to the extent there is a collective action problem, it's not between employees, employees of the Frontier AI labs, but between companies and countries that want the ability to slow down. Um, and Neil is kind of raising the question here of why uh wasn't this petition being led by the Frontier Labs themselves? Why is it being led instead by um employees at these frontier labs? I don't know if you have any thoughts about that. I >> I mean, you know, it's hard to say for certain. I I do think that um you know, there are different circumstances that apply to the companies versus the employees of the companies. So, one, we know that because of how competitive it is for the the the top tier of AI talent that that talent is often given um a high degree of discretion um and the ability to make public statements that maybe is not true for other industries. And so they're allowed to say what they feel and so they might be more vocal than we might see in other industries. But you know companies they have had um you know tens or hundreds of billions of dollars in investment and they do owe you know they've made certain um uh commitments to those uh venture capitalists who have who have funded them and so I think that that does put some constraints on their ability to do things even though I think you know what we've seen is that um open a is very upfront that it has this um complicated governance structure where it's nonprofit makes many of the controlling decisions for what the company can do and Anthropic is organized as a public benefit corporation and so they've told a lot of investors that we will sometimes make decisions that are uh what we think are best for the world and not necessarily what's um best for the company. Um, I still think that there are more constraints on their ability to sort of signal things that might put the companies at existential risk than is true for individual employees. So, I suspect that plays a little bit into it. >> Yeah. And that makes a lot of sense. Um, now zooming out a bit from this uh petition specifically to this uh question of AI safety and uh cyber security in general. I'm curious how are people in AI policy and the cyber security communities responding to these revelations? First uh the incidents disclosed by OpenAI and now by anthropic. Yeah. So I think it is um definitely be interpreted by a significant part of the AI community as this is a harbinger of things to come. So I don't think anyone thinks that we've reached some sort of high watermark and that uh incidents like this are going to become less common. Um in in fact I think this really raises the the concern that sort of as models become more capable um that they will increasingly engage in things that that feel like they're coming straight from science fiction. um and that um that right now we're not seeing any significant slowdown in how models are advancing um and we're not seeing uh a circumstance where the AI tools have improved our cyber defenses so much that this is no longer a concern. If anything, it feels like the ability for models to to do to find vulnerabilities and evade protections and escape containment measures is increasing faster than we know how to use these tools to sort of stop th those sorts of things. Um, and so I think this is raising some debate about, you know, like what is the root of the problem and, you know, where should we invest resources or where should we think about policy interventions? Um and I think there there are two broad camps here. I don't think they're mutually exclusive and and almost certainly both of these are true to some extent. So the one is um saying this is just a basic cyber security issue that if we had better sandboxes, if we had better cyber security, if we were um employing best practices consistently across the industry, then something like the open air incident would not have happened because it wouldn't have been possible to escape the sandbox. And so what we really need to do is figure out how to be better at detecting vulnerabilities and fixing them. um patching um and really making sure that the systems we um put these models in are really buttoned up so that there's no possibility of this happening. Uh the other uh uh camp is that um that this is, you know, perhaps a a foolish thing to try to do because historically we've seen that trying to make a perfect system that is perfectly secured is really really difficult. And that was even before the era of really powerful tools that could scan and find vulnerabilities quickly. And so the thing we really need to do is invest in alignment. That is making sure that we are fundamentally programming these models to do the right thing for for some definition of right thing to follow user intent to not engage in things that are um problematic that could lead to harm either financial harm or physical harm. And so that that what we should really do is invest more resources in the kind of basic research that would allow us to make progress on this alignment issue. And sort of what that means is that we would fundamentally have models that we could give open-ended instructions to. and they would be like, "We're going to do the best we can to solve this problem, but we're not going to do it in a way that sort of escapes our in uh sort of contravenes our instructions or or potentially puts systems at harm or that does things that we sort of suspect are clearly unauthorized or by the the people who gave us those instructions. >> Right. And there's an interesting link here from the between the cyber security camp, the people who think um cyber security is is the core issue here and then the people advocating for more openweight model development. Um this is coming from hugging face CEO uh Clem Delay. Hopefully I'm saying that correctly. Um after the incident with OpenAI's model escaping containment, he said that AI safety won't be solved by any single company working in secret, it will be solved in the open collaboratively with broad access to AI for every defender everywhere. And that's kind of a reference to um the role that open models played in Hugging Face's um response to to the incident. Um and also kind of wrapped up in the midst of this all these stories. Um on July 24th, Nvidia published an open letter to US policy makers titled open weights in American AI leadership in which it makes the case for openweight models as the foundation of an AIdriven economy. And part of Nvidia's core argument here is that open models help strengthen AI safety while closed models jeopardize it. Uh the letter states, quote, "Openess may be one of the most important paths to AI safety and security. Relying solely on closed models is not inherently safe as they cannot be they cannot be breached, misused, or fail in out in ways the outsiders cannot detect. And concentrating advanced AI capabilities behind a small number of closed models compounds that risk. It results in a small number of single fa points of failure, weakens competition, and leaves critical technology in the hands of a few providers. So, I'm wondering if you can speak a little more about that connection between AI safety and open weight model development. Um, how much merit do you think there is to that argument that Nvidia is laying out? And there's certainly merit to this idea that um openw weightight models because they're open um they're widely distributed allow for like this this collective body of knowledge to form. These models often get you know more scrutiny than closed models. So there's definitely an element of truth to this. Um but there but there are sort of unique risks posed by uh open models as well. Um, and there's a real sort of balance here. And I don't think that we have currently figured out a good way to like empirically assess this, right? So, we're we're sort of muddling our way through. A lot of this will h depend on what the overall trajectory of AI development is. Some of this will depend on specifically how um open weights uh how these models develop. And then um some of it will develop uh some of it will depend on how we figure out how to deal with the cyber security issues raised by bottles. And in particular like there is this argument that as AI models get better that they'll they'll help defenders more than attackers and they'll shift the balance away from attackers towards defenders. I think uh you know that's an argument that's been made but maybe like there's increasing skepticism that's that's actually how it's going to play out. Um and I think all of these are are important to figuring out like what the the overall impact of open weight models is but you know basically the argument is if you have an open model you will definitely be able to scrutinize it more people will be able to look at it more people will be able to uh interrogate it in really really deep ways. um you can you you can fine-tune it, see how to circumvent um safeguards, uh all all sorts of ways of scrutinizing that model and and a lot of that will can and will be published publicly and this is very different than closed models where the because they're closed they're served through APIs um companies will be able to control how they're used to some extent both through technical measures and through terms of uh service measures um and that often times they'll sort of make agreements with organizations to test those but not all that information will be publicly released. Um but there is a there's a flip side here which is you know typically open weight models they come with fewer safeguards built in. um those safeguards can often be sort of removed. The safeguards that do exist can often be removed uh relatively easy with uh very little data or technical knowledge or um token use involved. um that uh one of the concerns we're really thinking about here is that there are there's a possibility that open any model can have sort of an asymmetric backdoor that allows you to get it to do things you don't want like a form of jailbreaking that allows you to like bypass all the safeguards that's in the model and that those can be programmed in um and very easy to activate if you know the right way to do it but very hard to detect. um if you don't uh you know you might think of it an analogy to sort of like cryptography where it's very easy to decode something if you know the key but if you don't know the key figuring out is incredibly difficult um and if that kind of thing exists we don't have a lot of evidence for it right now not a lot of concrete evidence but if that sort of thing exists then even if a model is released openly and millions of people are testing it they may they may just not be able to fight that functionality. Um, >> and if that's the case, the then you then you run into the biggest issue with open models, which is that um once they're out, they're out and you can't do anything to control who has access to them or to take steps to sort of if they have a really dangerous capability, stop people from exploiting that dangerous capability. Um what we saw previously is the U when the US government expo uh you know implemented an export control on anthropic and said you can't provide your model to foreign nationals. They were able to go and flip the switch and turn that off and so then people couldn't access it and that's because they controlled it. But you couldn't do that with a model that's available on the internet for anyone to download. And so this is the trade-off that we're really trying to think about and and deal with. And like I said, I don't think we necessarily have figured out a good way to to trade those off yet. And so we're we're sort of muddling our way through. >> Right. Well, this is the AI policy podcast. So I'd like to wrap up this conversation uh by talking about policy. Uh could you walk us through what are the uh AI policy implications for um all of these stories? Yeah. So I think there are um there are a couple of things that we should really think about this and that is that you know like for me the issue of open models has always been one of the um really difficult maybe the most difficult policy issue that uh governments have to grapple with and one of the ways they'll have to grapple with it is sort of the various frameworks they have for testing and release of models will need to incorporate open models in some way. As long as those models are sort of, you know, weaker than the frontier, um, you know, not on par with the most powerful models, uh, we probably didn't need to address it explicitly. But the closer they get to the frontier, the closer um or the more deliberate we'll need to be about whether uh there should be any exceptions to the kinds of um frameworks we're thinking about relating to testing and release of frontier models generally. And so what that means is um you know for example the the White House is sort of working on and has supposedly finalized a volunteer frontier AI model review process. Um it it's easy to think about how they could include open models in that process. It's harder to think about well what happens if a model is released and then we we find some significant concern about it. What do we do then? So, I think that this is going to cause a lot of um places in the US and around the world to think a lot harder about what it means to regulate open models um and what you can do uh to sort of control access to them or sort of make sure they're safe um because once they're out, they're out and there's no sort of putting the genie back in the bottle. Um I think the the other things that we should um really think about uh these are are going to extend on things we talked about um in the past. One is that a lot of the frameworks we're thinking about for regulating AI really have to do with models that are about to be released to the public. And it's very natural to do this because there is a very defined point at which you want to test the model. it's easy to figure out like what checkpoint you should be thinking about. Um the timelines are more clear when it comes to these internal deployments, right? Some of these models um may intended to be be released publicly, but that's not a guarantee, right? Some of these models could be uh for internal testing only or they might be intended for use only within the company. And now I think we have to think a lot harder about extending our uh regulation or test how we're thinking about testing to happen much earlier in the in the process and to include uh some of these internal only models. And it turns out that this is a lot harder to do, right? you have a lot more questions about when you want to do this and like what models would fall under scope and how the the the government who might be interested in these models would even know they exist, right? A lot of these are going to implicate proprietary information. There's going to be a lot of experimental models. We could be thinking about a a a far greater number of models than are um implicated when you're just talking about models intended to be released to the public. And so this is um an issue I think also where we're going to see a lot of discussion and a lot more sort of um exploration of various policy options. Um and then the the third thing has to do with the fact that hugging face because of the various safety controls placed on models from open ananthropic had to do some of its cyber security testing uh around the open AAI incident um using open models using Chinese models because they didn't have the same safeguards in place and so the those models didn't get overzealous and say hey you mentioned the word cyber security and therefore we're going to refuse to answer this question. Um and so I think um we should also be figuring out how can we get trusted access uh trusted actors access to the types of models that would allow them to improve their cyber security practices. So really like segment out there's a sort of general capabilities that we don't want available to a broad segment of the population because that's just too much risk surface. Um, but there are definitely organizations that we trust and that have very serious um, and significant cyber security issues that they want to proactively address and how can we get them the right tools uh, to allow them to to improve their cyber security practices. >> Right. Well, that feels like a great place to wrap up. Um, thank you Aloque for your insights and thanks as always to our audience for tuning in. >> All right. Thanks, >> [music] >> Thanks for listening to this episode of the AI Policy Podcast. If you like what you heard, there's an easy way for you to help us. Please give us a five-star review on your favorite podcast platform. Subscribe and tell your friends. It really helps when you spread the word. This podcast was produced by Sarah Baker and Matt Mand. See you next time.