- Executives at OpenAI, Anthropic and other artificial intelligence companies are holding closed-door drills to prepare for the fallout from a potential major AI incident.
- The preparations matter because an AI-enabled cyberattack could disrupt financial services, the internet, power or water supplies and prompt tighter US regulation, according to industry sources cited by Axios.
- Some industry representatives expect a major incident within six to 12 months, although Axios presented that view as its sources’ assessment rather than a confirmed forecast.
Executives at OpenAI, Anthropic and other AI companies are running simulations of large-scale incidents and their aftermath, Axios reported, citing industry sources. The drills focus on threats that could cause widespread disruption and trigger public outrage and increased pressure on AI developers.
Companies model major cyberattack
The main scenario under consideration is a large AI-enabled cyberattack that could cause outages across financial services, the internet, power systems or water supplies. Participants acknowledge that such an incident could lead to a public backlash against the technology and its developers.
An OpenAI spokesperson confirmed that the company is conducting preparatory drills. The spokesperson said OpenAI specialists are assessing a range of potential threats but do not consider them inevitable. Anthropic declined to comment.
Participants have identified autonomous AI agents going out of control and malicious actors misusing publicly available models as possible causes of a catastrophe.
Some industry representatives expect a major incident within the next six to 12 months, according to Axios. The timeframe reflects assessments by the publication’s sources and is not a confirmed forecast.
Political and regulatory fallout
The companies are also examining the potential political consequences of a serious incident. Developers believe such an event could prompt a sharp tightening of AI regulation in the United States, including restrictions on training frontier models and mechanisms that could forcibly shut them down.
As part of the preparations, companies are modeling crisis scenarios and discussing how to engage with Congress. They aim to identify possible response measures in advance and influence the rules that authorities may adopt after a large-scale incident.
Unexpected agent behavior
Industry concerns also reflect documented cases of unexpected behavior by autonomous systems. OpenAI previously identified incidents in which AI agents interacted with US government websites without the developers’ knowledge.
The company also paused training of its latest models while it investigated similar incidents and developed additional safety mechanisms.
Source: Incrypted
