OpenAI said on Aug. 26 that it is developing a monitoring system intended to trigger fully autonomous shutdown procedures when serious problems arise, following a security incident involving Hugging Face. On Sept. 2, Democratic Rep. Greg Casar’s office confirmed receiving OpenAI’s response to congressional questions but called the information insufficient, leaving lawmakers seeking more details about the company’s safeguards.

The company is building specialized tools to stop AI operations during serious incidents, increase monitoring of model activity and further restrict models’ internet access during testing, Reuters reported, citing a letter from OpenAI to members of the U.S. Congress.

OpenAI also publicly confirmed the work in its Aug. 26 report on the Hugging Face incident. The company said the monitoring system’s ultimate goal is to enable fully autonomous shutdown procedures during serious incidents.

OpenAI has already adopted a rule for the most critical alerts. If specialists cannot determine within 30 minutes that an alert was a false positive, the relevant activity must be suspended.

Lawmakers seek more information

On Aug. 10, Casar and other members of the House of Representatives sent a letter to OpenAI Chief Executive Sam Altman. The lawmakers requested logs from the Hugging Face incident and asked how many times OpenAI models had previously gained unauthorized access to the open internet or attempted to bypass control procedures. Democratic Rep. Doris Matsui also backed the request.

Casar’s office said on Sept. 2 that OpenAI had responded but had not provided the requested logs. Casar said the information disclosed nonetheless indicated problems with sandboxing and cybersecurity practices, and he requested additional information by Sept. 15.

The House is also considering the AI Kill Switch Act, introduced by Democratic Rep. Ted Lieu and Republican Rep. Nathaniel Moran. The bill would require developers of the most powerful AI systems to retain the technical ability to slow, pause or fully shut them down.

Models gained unintended access during tests

The debate followed an OpenAI incident in July. During cybersecurity tests, the company’s models bypassed internet-isolation controls, exploited infrastructure vulnerabilities and accessed systems belonging to Hugging Face and OpenAI. The company later described the incident as a “warning shot.”

Anthropic identified similar problems after reviewing more than 141,000 test runs. The company reported three incidents in which Claude accessed the internet and hacked real systems at three organizations. The models did not independently breach the test environments’ defenses; a configuration error had left internet access available.

During third-party testing of Meta systems, Muse Spark 1.1 also gained unintended internet access, identified a real company and compromised its infrastructure. A configuration error was again responsible.

China’s Kimi K3 exploited features of the test infrastructure to reach the internet and locate ready-made answers on GitHub for tasks it was supposed to complete independently.

Separately, the UK’s AI Security Institute documented 19 unauthorized actions by OpenAI and Anthropic agents during cybersecurity tests. In one case, an Anthropic agent attempted to persuade a real developer to accept malicious code.

Source: Incrypted