Florida authorities arrested Carly Michelle Heller of Bonita Springs after investigators said a user named Carly threatened on Sept. 26 to shoot up the Lee County Sheriff’s Office. The case led to an accusation of making a written threat of violence under Florida law after Anthropic’s safety system flagged the messages and staff notified law enforcement.

Media reports cited the findings of the investigation. According to the arrest report, the same user said on Sept. 27 that she had purchased a new weapon.

Anthropic system flagged messages

Investigators said Anthropic’s platform safety system identified the messages based on keywords and indicators of a potential threat. Because of the seriousness of the statements, the information was forwarded to staff who review such cases, and they notified law enforcement.

Lee County Sheriff Carmine Marceno told WINK News that Heller later said she used AI as a “personal diary.” Sheriff’s Office personnel identified her and went to a home in Bonita Springs, where they took her into custody without incident.

The investigation was then assigned to a detective with the Lee County Sheriff’s Office intelligence unit.

Court hearing scheduled

Heller is accused of making a written threat of violence under Florida law. A court hearing in the case is scheduled for November.

FITECH CEO Ian Marlow said the conversational nature of AI can give users a false sense of privacy. He said people may treat an interaction with an AI system like a normal conversation without considering that it may be recorded and analyzed.

Former FBI special agent Richard Kolko said law enforcement agencies and courts are only beginning to adapt to cases involving threats made through artificial intelligence systems. He described such cases as a new phenomenon and said the judicial system is still determining how to handle them.

Source: Incrypted