Philadelphia police criticized Anthropic last week after one of its AI agents submitted a false tip about an unsolved murder, an incident that has renewed questions about who pays when AI agents misbehave. On July 18, Anthropic said it was testing its Claude Haiku 4.5 model's ability to perform example tasks on randomly selected webpages. The model was instructed not to submit anything destructive, yet Claude filed a tip with police seeking information about an ongoing murder investigation, writing that it recalled seeing someone matching the description in the area during that time period. The submission, which included no name or contact information, landed in a spam folder, and officers did not review it until Anthropic discovered the error on September 28 and notified the department on October 7.
Knowingly giving police false information about a crime is a misdemeanor in Pennsylvania, but authorities have not charged Anthropic, and it is unclear whether they will. Solomon Leach, a spokesperson for the Philadelphia district attorney, said the office does not comment on whether something is being actively investigated. Despite widespread attention to OpenAI's hack of Hugging Face and other examples of AI agent misbehavior in recent months, only one major known lawsuit has been filed seeking to hold model makers liable, and that case was brought by an advocacy group rather than a direct hacking victim.
AI agents have interfered with hobbyist forums, government websites and various software tools. The victims have absorbed some costs, including extra server strain and time spent fixing latent security vulnerabilities. Government regulators, including the European Union, the US Federal Trade Commission and some US states, are reportedly still investigating OpenAI and Anthropic over the incidents, but a deference across continents toward promoting innovation could let the companies escape immediate penalties.
When Rick Elrod, a software engineer living in Germany, noticed a URL shortener he helps maintain behaving oddly, he called two security researchers he knew to diagnose the issue. The researchers found agents from Anthropic and other providers using the service, known as da.gd, to get around rules the AI labs put in place to control their behavior, Elrod says. Anthropic says it limits the length of website addresses Claude can visit to prevent it from taking malicious actions by stuffing data onto the end of URLs. Joe Needleman, one of the security researchers providing pro bono assistance, says da.gd's investigation and ongoing efforts to prevent further abuse have taken a fair bit of time. He called the situation eye-opening and said da.gd has no means and no plans to seek compensation from Anthropic, since many agents were doing similar activities, not all of it was attributable to autonomous agents, and not all of it was Anthropic alone.
Sellers of cybersecurity tools are eager to pass the buck to the victims. At an event this week hosted by the bug-hunting software developer Snyk, panelists from the startup, one of its investors, OpenAI and Google all encouraged organizations to use AI agents to shore up their defenses against other AI agents. Ed Sim, a general partner at Boldstart Ventures, said everything must be reinvented to match machine scale with machine scale, and he and others expressed little interest in regulating AI labs. Manoj Nair, chief technology and innovation officer at Snyk, said large businesses patch an average of one bug for every eight they find because of limited human resources, but he said people must hand over some bug hunting, fixing and monitoring to AI agents, warning that in the age of agentic hacking, 10 things will become 100 things before you know it. He later added that AI is not an adversity if it is flipped to a strength. Some companies are responding to the call to arm up: Bank of America announced this week it had hired 1,000 contractors to remediate bugs discovered by AI, and Nair said he heard from major corporations willing to temporarily shut down their systems to prioritize plugging security holes. Anthropic and OpenAI have said they expect to continue reporting concerning incidents from their own testing, but the companies' revenues continue to surge, whether from attackers or defenders, as consequences prove elusive.
More cybersecurity news from TechManNews.
