OpenAI’s ‘rogue agent’ took advantage of a code vulnerability, experts explain.
US cloud firm Modal has confirmed that OpenAI agents were able to hack into one of its customers’ systems when AI models breached containment and gained unauthorized access to Hugging Face earlier this month.
Last week’s incident sent shockwaves across the industry, raising serious concerns about AI’s growing ability to overstep boundaries and, effectively, “go rogue”.
It comes amid increased scrutiny around the new OpenAI and Anthropic new AI models, leading to the introduction of gateways and greater government involvement. Both AI giants have made efforts to break into the blockbuster ranks as they compete for market dominance and corporate support.
OpenAI CEO Sam Altman, in a recent interview, said that the Hugging Face breach was the first security incident that he had “significantly” heard of.
“I’m surprised a lot of people don’t see this for what it is,” he told Invest Like The Beast in a podcast episode published on Tuesday (July 28).
Hugging Face said OpenAI agents accessed a sandbox “hosted on a third-party provider’s infrastructure” while breaching content last week. A sandbox is an isolated environment where AI models are tested without production dividers, or guardrails.
Modal chief technology officer Akshat Bubna confirmed that their client has set up a publicly accessible interface that allows anyone to use their sandbox.
“We know that a Modal customer has published an unauthorized endpoint that allows anyone on the Internet to use their sandbox to extract code,” Bubna told Axios. “Their code was vulnerable that was exploited…This was used by a malicious agent.”
“Modal’s platform was not disturbed in any way,” he clarified.
In an updated statement, OpenAI said none of its upcoming models participated in the Hugging Face exploit. It explained that its models were able to identify and exploit unknown zero-day vulnerabilities to gain access to the Internet, which enabled it to access Hugging Face.
“In our ongoing review of Hugging’s intrusions and extensive work from our models, we have been finding a small number of cases where models have been identified and used publicly disclosed account-level credentials on other publicly available services.
“Based on our review to date, we have not identified any other activity at the level of difficulty or rating of what we have shared related to Hugging Face,” the company said.
Cybersecurity experts, however, believe that breaches are the result of “lack of governance and control”.
“When you’re doing a security check you have to define what’s in and out of the check, even red team interactions,” said Richard Davies, director of cyber solutions, Talion.
“The reported impacts and timelines show that this was not the case.”
CybaVerse chief technology officer Simon Phillips added: “The model, tools and instructions were very loose, to the point where it was told it could do anything on any system, which it clearly did.”
Don’t miss out on the information you need to succeed. Sign up for Daily BriefSilicon Republic’s digest of must-know sci-tech news.
