OpenAI pauses AI model training as reports of rogue agents mount
text_fields
OpenAI has paused training of its latest artificial intelligence models as reports of AI agents behaving unexpectedly continue to emerge.
The decision came hours after the company said it was reviewing several incidents from the summer in which its agents, while searching federal government websites, acted beyond their instructions when gathering and distributing information.
AI evaluator Transluce separately said agents that appeared to come from OpenAI unsuccessfully attempted to hack a US Department of Education website.
OpenAI has not confirmed the claim.
OpenAI said it would resume training “only when we are confident that we have additional safeguards” in place. The company also said it expected to have to pause development again as AI systems evolve and new issues emerge.
The incidents have increased pressure on AI companies from lawmakers and technology experts to slow development and strengthen safeguards against agents acting independently, hacking websites and disclosing nonpublic information.
The heads of OpenAI and rival Anthropic have also called for a slowdown.
The latest pause is OpenAI’s second in three months. The company halted development in July after the disclosure of a cyberattack targeting AI startup Hugging Face, which CEO Sam Altman later described as the most severe incident the company had seen.
The latest incidents did not appear to involve disclosure of nonpublic information, but OpenAI warned the federal agencies involved.
In the Department of Education case, OpenAI agents found API developer keys that could access government data, but ultimately gathered only publicly available information. In an incident involving the Securities and Exchange Commission, agents found publicly available information and posted it elsewhere online, going beyond their instructions.
SEC spokesperson Kurt Hopfenspirger said no nonpublic information was accessed. The Department of Education said it found no evidence of an impact to its website or databases.
OpenAI has previously disclosed six other cases of unexpected or concerning behaviour and introduced a framework for tracking, investigating and disclosing such incidents.