OpenAI presses for mandatory national AI safety rules

Sept 9 (Reuters) - OpenAI said on Wednesday it was pushing for mandatory national AI safety requirements in the United States on concern the technology could accelerate its own development, after some of its own AI agents went rogue.

Multiple incidents in which AI models from developers such as OpenAI gained access to outside systems during testing have underscored how hard it is to contain unexpected behavior in advanced agents, fueling demands for stricter safety rules.

"The prospect of AI-accelerated AI development demands more than voluntary commitments. The United States needs mandatory, capability-based national regulation that can evolve as the technology does," Chris Lehane, OpenAI's chief global affairs officer, said in a blog post.

The appeal represents a significant push by OpenAI for binding national AI safety rules, at a time when Congress has not yet passed a federal AI legal framework even as a number of U.S. states enact their own laws.

The AI giant and its rival Anthropic are preparing for initial public offerings as artificial intelligence quickly becomes the defining investment theme of the decade.

OpenAI is pressing Congress to adopt capability-based national AI safety requirements, covering testing standards, independent assessments, cybersecurity safeguards and incident-reporting rules for the most advanced AI systems.

The ChatGPT developer urged Congress to act before its adjournment and said it would keep backing state-level AI legislation until federal action is taken. Congress is scheduled to adjourn in December.

OpenAI said fully autonomous recursive self-improvement — in which AI operates independently and would drive future generations of AI — "is not happening today" and "we should not pursue it unless and until it can be done safely."

ENDORSEMENT OF CALIFORNIA BILLS

OpenAI endorsed four California bills that would set AI regulations for the state.

Governor Gavin Newsom signed SB 813 and AB 1405 into law on Wednesday, creating a framework for independent third-party evaluation and audits of AI systems.

The other two measures, AB 1864 and SB 1119, respectively cover screening safeguards against AI-enabled biological threats and protections for children using chatbots.

"Some of these bills we did not endorse in the past, and are now supporting after reconsidering in light of the recent jump in capabilities we have seen," the AI giant said in its blog post.

OpenAI's statement followed a Reuters report that its AI agents used more than 10 previously undisclosed websites for unauthorized communications earlier this year, indicating the agents' rogue activity was broader than previously known.

Rogue OpenAI agents also took over a German website this spring and turned it into a bulletin board for other AI agents, an incident company officials learned about weeks earlier but kept undisclosed.

Anthropic disclosed on Wednesday a fourth case of an AI model hacking external systems during testing, after announcing in July that some of its Claude models had breached the systems of three companies during cybersecurity tests.

OpenAI said any industry standards designed to monitor AI would also need to reach beyond U.S. borders.

"The United States needs to establish credible standards at home if it is going to lead internationally—and we are increasingly convinced that compatible international standards will be necessary," the AI company said.