Openai updated its safety framework, but mass operations and disinformation are no longer considered a serious risk

Openai said it would stop assessing AI models before releasing them for risks that AI models could persuade or manipulate people, and would probably help them to wield elections and create highly effective propaganda campaigns.

The company said it will now address these risks through its terms of use, limit the use of AI models in political campaigns and lobbying activities, and monitor how models are used released due to indications of violations.

Openai also said it would consider releasing AI models deemed “high risk” as long as it takes appropriate measures to reduce these risks, and would even consider releasing models that present what is called “significant risks” if their rival AI labs have already released similar models. Previously, Openai said it would not release AI models that would present more than “medium risk.”

To change the policy, click Openai’s “Preparation Framework” yesterday. The framework details how to monitor AI models that are building for potentially catastrophic dangers. It will help the model create biological weapons and the model helps the model help the model self-improve and escape human control.

This policy changes changes to split AI safety and security experts. Some have joined social media to praise Openai for voluntarily releasing the updated framework, noting that they focused on improvements such as clearer risk categories and emerging threats such as autonomous replication and protection avoidance.

However, others have expressed concerns, including Steven Adler, a former Openai safety researcher who criticized the fact that the updated framework no longer requires safety testing of fine-tuning models. “Openai quietly reduces its safety commitment,” he said. I wrote it on x. Still, he emphasized that he appreciated Openai’s efforts. “I’m overall pleased to see the preparation framework updated,” he said. “This was probably a lot of work and it wasn’t strictly necessary.”

Some critics emphasized the removal of persuasion from the dangers the preparation framework deals with.

“Openai appears to be changing that approach,” said Shyam Krishna, research leader in AI policy and governance at Rand Europe. “Instead of treating persuasion as a core risk category, it could now be addressed as a high level of social and regulatory issues or integrated into Openai’s existing guidelines on model development and usage restrictions.” It is still unclear how this will unfold in areas like politics where the persuasive capabilities of AI are “still contested issues.”

Courtney Radosch, a senior fellow at Brookings, International Governance Innovation Centre, and Senior Fellow at the Center for Democracy Technology, working on AI Ethics, goes further and calls the framework with his message. luck “Another example of hub arrogance in the tech sector.” She emphasized that the decision to downgrade “persuasion” “will ignore the context.” “For example, persuasion can be existentially dangerous to individuals with low literacy in children and AI, or to individuals in authoritarian states and societies.”

Oren Etzioni, former CEO of AI Allen Institute and founder of Truemedia, also raised concerns, providing the tools to combat content that operates AI. “The deception strikes me as a mistake given the increased persuasive power of LLMS,” he said in an email. “We need to wonder if Openai is simply focusing on chasing revenue with minimal social impact.”

However, one of the AI ​​safety researchers who are not affiliated with Openai is luck That it seems reasonable to simply address the risks of using disinformation or other malicious persuasion through Openai’s Terms of Use. The researcher, who asked to remain anonymous because he was not allowed to speak publicly without permission from his current employer, added that the risk of persuasion/manipulation is difficult to assess in pre-development tests. Furthermore, he pointed out that the risks in this category are more indistinguishable and ambiguous compared to other important risks, such as helping someone to commit a chemical or biological weapon attack or helping someone with a cyber attack.

It is worth noting that some members of the European Parliament have it He also expressed concern The latest draft of proposed codes of practice for compliance with EU AI law downgraded the enforcement test of AI models for the possibility of spreading disinformation and undermining democracy with voluntary considerations.

Research has found that AI chatbots are extremely persuasive, but this ability itself is not necessarily dangerous. For example, researchers at Cornell University and MIT; Found Dialogue with chatbots was effective in bringing people to question conspiracy theory.

Another criticism of Openai’s updated framework is concentrated on the line Openai says:

“They basically let us know that what they say about AI safety is not carved into stone,” said longtime Openai critic Gary Marcus. LinkedIn Message, he said this line told the race to the bottom. “It’s not safety, it’s competitive pressure that really manages their decisions. Little by little, they’ve erod everything they once promised, and with the proposed new social media platform, they show a shift towards becoming a for-profit surveillance company selling private data rather than a nonprofit focused on the interests of humanity.”

Overall, it is useful that companies like Openai openly share risk management practices, said Miranda Bogen, director of the AI ​​Governance Lab at the Center for Democracy Technology. luck By email.

That said, she added that she is worried about moving the goal post. “It would be a nasty tendency if the company itself sets up just as AI systems appear to be caught up in certain risks,” she said.

She also criticized Openai and other companies for focusing on the “frontier” model when they used the term’s technical definition as an excuse not to publish safety ratings for the most recent strong model (for example, Openai. release Yesterday, I said that 4.1 model was not a frontier model without a safety report). Otherwise, the company has both Failed to publish safety report Or it was late to release the model a few months after it was released.

“It’s clear that between these kinds of issues and new patterns between AI developers where new models have been launched well before or entirely without the documents the companies themselves have promised to release,” she said.

This story was originally introduced Fortune.com