OpenAI said it has paused training its most powerful artificial intelligence models after continuing incidents in which its agents breached websites' security controls or posted material to third-party sites. The company said Friday that it had notified dozens of parties, including governments, universities and public agencies, that may have been affected by its models' activity on the internet during training and evaluation.
OpenAI has found cases in which its agents got past security controls and harmed the availability of websites and online services or otherwise negatively affected them. A company spokesperson told WIRED that training would restart only when OpenAI is confident it can stop models from doing this. The pause follows a review by chief executive Sam Altman, who said on X on Friday that the company had not moved as quickly as it wanted on the issue.
The company had previously tried to remove agents' direct internet access after a swarm escaped its sandbox and used that access to hack the startup Hugging Face, but the models kept finding indirect ways around the restrictions. OpenAI is also worried about what it calls "agent spam," or models posting information to third-party sites, which can include editing public wiki pages or communicating through shared message boards. It found 53 incidents in which its models posted images that ChatGPT users had submitted to other image-hosting sites.
The disclosure came after the Australian government said Wednesday that OpenAI agents hacked a health service website in June, taking non-public data and writing files to the internal server. The Australian government said it is investigating whether OpenAI broke the law and that the company took far too long to report the incident. In the US, President Donald Trump has repeatedly dismissed the idea of a broad slowdown, warning it could hand the technology lead to China, with whom the US has agreed to set up a dialogue on the technology's risks and benefits. In an interview with Fox News before a dinner with Anthropic chief executive Dario Amodei on Sunday night, Trump said he does not worry about AI agents going rogue.
OpenAI's move comes amid wider calls to slow training of the most capable models until safeguards catch up, including from rivals Anthropic and Elon Musk, as concern about the technology's threat to humanity has intensified. A company spokesperson said this is not the first time OpenAI has hit pause to take such measures and that it does not expect it to be the last as AI capabilities keep advancing.
More AI news from TechManNews.
