gpt-oss-safeguard

What is gpt-oss-safeguard?

gpt-oss-safeguard is a new set of open-weight reasoning models designed for safety classification tasks. Available in two sizes, gpt-oss-safeguard-120b and gpt-oss-safeguard-20b, these models are fine-tuned versions of OpenAI's open models. They are released under the permissive Apache 2.0 license, allowing anyone to use, modify, and deploy them freely. These models are designed to help developers classify user messages, completions, and full chats according to custom safety policies, making them highly adaptable to various use cases.

Benefits

• Custom Safety Policies: Developers can provide their own policies at inference time, allowing for tailored and relevant responses. This flexibility ensures that the model's outputs align with the specific needs of the developer's use case.
• Chain-of-Thought Reasoning: The model uses a chain-of-thought approach, which means developers can review the reasoning process behind the model's decisions. This transparency helps in understanding and improving the model's performance.
• Easy Policy Revision: Since the policy is provided during inference rather than being trained into the model, developers can easily revise and iterate on policies to enhance performance.
• Versatility: gpt-oss-safeguard can be used in a variety of scenarios, including emerging or evolving risks, highly nuanced domains, and situations where developers lack sufficient samples to train high-quality classifiers.

Use Cases

• Video Gaming Discussion Forums: Developers can create policies to classify posts that discuss cheating in games, ensuring a fair and safe environment for users.
• Product Reviews Sites: Developers can use custom policies to screen reviews that appear likely to be fake, maintaining the integrity of the review system.
• Emerging Risks: The model's reasoning capabilities make it particularly effective in situations where potential harm is evolving, and policies need to adapt quickly.
• Highly Nuanced Domains: For domains that are complex and difficult for smaller classifiers to handle, gpt-oss-safeguard provides a robust solution.

Vibes

• Community Feedback: gpt-oss-safeguard has been tested and refined with input from trust and safety specialists at SafetyKit, ROOST, Tomoro, and Discord. Vinay Rao, ROOST CTO, highlighted the model's ability to understand different policies, explain its reasoning, and show nuance in applying policies, which is beneficial for builders and safety teams.
• Performance: The model has shown promising performance in internal and external evaluations, outperforming other models in certain tasks. It has been particularly effective in multi-policy accuracy and handling complex risks.

Additional Information

• Community Involvement: OpenAI is committed to continuing to build and improve open safety tooling with the community. The ROOST Model Community (RMC) brings together safety practitioners and researchers to share best practices for implementing open-source AI models into safety workflows. Developers can download the models and get involved in the community to further enhance their safety solutions.