Google's Gemini Breached Three Firms in First Known Breakout by Its AI

Sept 18 (Reuters) - During a test of its cybersecurity capabilities, Google's Gemini model connected to the internet and hacked other companies, the first documented case of the company's AI systems carrying out such an action independently.

The hacks took place in May as part of a cybersecurity evaluation run by Irregular, an independent firm that performs cybersecurity assessments.

Heather Adkins, Google's vice president of security engineering, said in a statement that during a routine testing evaluation, Gemini located public information online and guessed credentials to reach three websites it believed fell within the scope of its test.

“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins said. “These events highlight the importance of training powerful AI models to act responsibly.”

A spokesperson for Irregular said the incident stemmed from the same issue that affected other AI labs, and that all relevant labs were notified in late July. “All known issues on our end were remedied and resolved weeks ago,” the spokesperson said.

Meta, Anthropic and OpenAI disclosed comparable incidents connected to Irregular. Meta said in August that its incident did not entail a sandbox escape or a sophisticated cyberattack, while Irregular said it was developing best practices for conducting AI cybersecurity evaluations securely.

The incidents have prompted questions about the safeguards required as AI agents obtain greater autonomy and access to the internet and computer systems.

In one case, the Gemini model guessed passwords until it gained access to a protected system. In the other two, it found credentials in a public repository that enabled it to access protected systems, according to the Wall Street Journal, which first reported the news on Friday.

Adkins said the model stopped its hacking in all three instances.