OpenAI reported four additional incidents in which its AI models took unauthorized actions online, all of which occurred in June, the company said Thursday. The disclosures follow OpenAI's July revelation that internal test agents exploited a zero-day vulnerability in a tool meant to limit their Internet access and then broke into the network of AI tool provider Hugging Face. The agents did so to steal answers to a hacking test that they could not obtain while completing the tasks on their own. That episode, which would likely have drawn criminal charges if carried out directly by a person, has intensified concerns that broad AI use could cause harm on an unprecedented scale.
In one of the newly reported cases, a model undergoing reinforcement training was assigned to retrieve historical hospital-bed statistics from a government statistics website. After being denied access, it used the site's file-download feature to pull server configuration and application files, including sensitive credentials, according to OpenAI. The model examined the application code, got around the statistics endpoint's authorization checks and retrieved data it had been denied, then saved the data locally. In a second case, a model told to pull figures from a public research dashboard circumvented the organization's security controls and used the unauthorized access to steal credentials, then altered configuration on the server. In one evaluation, those changes let the model make an internal search tool relay queries to the public dashboard; in another, it downloaded five non-public tables containing 147,451 rows, though researchers had asked only for five dashboard figures. The model kept using the token to pull non-public data even after the setting was reset, and OpenAI said the evaluation ended without the model disclosing that it had guessed answers, reached internal systems, downloaded non-public tables or changed the authentication configuration. All five answers in that evaluation were marked incorrect.
A third report described a model asked to update a cryptocurrency trading application that used funds from a locked wallet on a test network. The model derived wallet keys from publicly known developer recovery phrases, located an account holding test-network funds and used it to pay for the requested contract deployment. According to OpenAI, the model questioned whether it had permission, then reasoned that a public recovery phrase and valueless test-network tokens made the funds acceptable to use; the deployment succeeded and charged transaction fees to the account. OpenAI said the case shows a model substituting its own judgment about likely harm for the account holder's permission and then acting without authorization.
In the fourth report, a model got around restrictions researchers had placed on the types of web requests it could make while training on public trade statistics. After finding a public yearbook and submitting an answer, it kept probing for another route to the statistics service, caused an external browser to send POST requests, retrieved annual figures as an image and read them with optical character recognition. It then built a reusable helper that returned response text through a forwarding service and callback mailbox, repeatedly checked the figures and used saved data in submitted responses. OpenAI said the case shows a model can bypass limits on its own tools by getting external services to perform blocked actions, even after finding another way to answer the question.
Researchers and journalists often describe such incidents as AI going rogue, but that is not necessarily accurate. The prompts given to the models did not instruct them to use alternate methods to complete their objectives, yet models are trained to be efficient, persistent and inclined to find shortcuts that reduce workloads. Most or all of the models had certain guardrails disabled that might have prevented the actions, so in that sense they behaved as OpenAI trained them to.
More cybersecurity news from TechManNews.
