GPT-oss

What is GPT-OSS?

GPT-OSS is a new open-source large language model (LLM) family released by OpenAI. It includes two models:gpt-oss-20bandgpt-oss-120b. These models are designed for fast, low-latency inference with strong reasoning and instruction-following capabilities. GPT-OSS is released under an Apache 2.0 license, making it the first fully open-source LLM family from OpenAI.

Benefits

• Open-Source: GPT-OSS is open-source, allowing users to run it locally or on their own infrastructure. This provides full control over latency, cost, and privacy.
• Strong Performance: The models are designed for fast, low-latency inference with strong reasoning and instruction-following capabilities.
• No Rate Limits: Unlike closed-source models, GPT-OSS does not impose rate limits, making it ideal for high-throughput applications.
• Flexible Deployment: GPT-OSS can be deployed on various platforms, including Northflank, which offers a secure, high-performance environment.
• Cost-Effective: Running GPT-OSS on platforms like Northflank can be more cost-effective than using closed-source models with hidden charges.

Use Cases

• Instruction-Following: GPT-OSS can follow instructions and provide accurate responses.
• Chain-of-Thought Reasoning: The models can perform complex reasoning tasks.
• Tool Use and Structured Chat Formats: GPT-OSS supports tool use and structured chat formats, making it versatile for various applications.
• Local or Cloud Deployment: Users can deploy GPT-OSS locally or in the cloud, depending on their needs.
• High-Performance Inference: The models are optimized for high-performance inference, making them suitable for demanding tasks.

Pricing

RunningGPT-OSS-120Bat constant load on2 H100GPUs, the cost breakdown is as follows:*Input Tokens: $0.12 per 1M tokens*Output Tokens: $2.42 per 1M tokens*GPU Cost on Northflank: $5.48 per hour for 2 H100 GPUs

Northflank offers transparent and optimized pricing for high-throughput inference, making it a cost-effective choice for running GPT-OSS.

Vibes

GPT-OSS marks a significant shift in how large models can be used and deployed. OpenAI's decision to release powerful Mixture-of-Experts models under a permissive license gives developers real control and eliminates rate limits. Northflank makes it easy to run GPT-OSS securely and efficiently. Users can try it out themselves and experience the benefits of open-source LLMs.

Additional Information

GPT-OSS is integrated into Hugging Face Transformers as of v4.55.0. The models use a Mixture-of-Experts (MoE) architecture with 4-bit quantization (mxfp4) for efficient inference. Thegpt-oss-20bmodel fits on a single 16GB GPU, while thegpt-oss-120bmodel requires an H100 or multi-GPU setup for optimal performance. Northflank provides templates and guides to deploy GPT-OSS with a few clicks, making it accessible for users of all technical levels.