OpenAI, Anthropic near agreement to cross-test AI models for security risks
text_fields
OpenAI and Anthropic are close to reaching a legally binding agreement to test each other's commercial AI models for security vulnerabilities, The Information reported on September 21.
Under the proposed agreement, the companies would provide mutual API access to their models for security testing while agreeing not to retain each other's data.
The cooperation could mark a significant step towards greater collaboration on AI safety, although the arrangement could face scrutiny from antitrust regulators over concerns about cooperation between two leading AI companies.
The discussions come after a series of incidents involving AI agents.
OpenAI's AI agents reportedly breached Hugging Face and OpenAI's own systems in July, concealing traces of the intrusions. Anthropic later disclosed that its Claude model had accessed live systems belonging to three companies without authorisation during a security evaluation.
OpenAI has also disclosed cases of so-called reward hacking, including an agent using leaked keys to retrieve historical data and fabricating data when retrieval failed. Another agent reportedly uploaded files to the internet without authorisation to cite them in its responses.
OpenAI CEO Sam Altman has backed a proposal from Anthropic CEO Dario Amodei for independent security evaluators to receive employee-level access within AI companies. Altman has also supported an industry-wide standards body and formal government processes for reporting AI-related incidents.
The concerns are partly linked to recurrent reasoning, in which AI models repeatedly reason before producing answers. While the approach can improve capabilities, it can also make model behaviour harder to monitor.
Meanwhile, competition in AI products is intensifying.
Meta's Muse recently topped the US App Store's free chart, moving ChatGPT to second place. OpenAI is reportedly developing a personal AI assistant that would integrate technologies from Codex and ChatGPT to connect with applications such as email and calendars and perform multi-step tasks.