OpenAI o1 and new tools for developers

OpenAI o1 and new tools for developers | OpenAI

OpenAI o1 and new tools for developers

Introducing OpenAI o1, Realtime API improvements, a new fine-tuning method and more for developers.

Share

Today we’re introducing more capable models, new tools for customization, and upgrades that improve performance, flexibility, and cost-efficiency for developers building with AI. This includes:

• OpenAI o1 in the API⁠, with support for function calling, developer messages, Structured Outputs, and vision capabilities.
• Realtime API updates⁠, including simple WebRTC integration, a 60% price reduction for GPT‑4o audio, and support for GPT‑4o mini at one-tenth of previous audio rates.
• Preference Fine-Tuning⁠, a new model customization technique that makes it easier to tailor models based on user and developer preferences.
• New Go and Java SDKs⁠ available in beta.

OpenAI o1 in the API

OpenAI o1⁠, our reasoning model designed to handle complex multi-step tasks with advanced accuracy, is rolling out to developers on usage tier 5⁠ in the API. o1 is the successor to OpenAI o1‑preview⁠, which developers have already used to build agentic applications to streamline customer support, optimize supply chain decisions, and forecast complex financial trends.

o1 is production-ready with key features to enable real-world use cases, including:

• Function calling⁠: Seamlessly connect o1 to external data and APIs.
• Structured Outputs⁠: Generate responses that reliably adhere to your custom JSON Schema.
• Developer messages: Specify instructions or context for the model to follow, such as defining tone, style and other behavioral guidance.
• Vision capabilities: Reason over images to unlock many more applications in science, manufacturing, or coding, where visual inputs matter.
• Lower latency: o1 uses on average 60% fewer reasoning tokens than o1‑preview for a given request.
• A new reasoning_effort API parameter allows you to control how long the model thinks before answering.

The snapshot of o1 we’re shipping todayo1‑2024‑12‑17 is a new post-trained version of the model we released in ChatGPT two weeks ago. It improves on areas of model behavior based on feedback, while maintaining the frontier capabilities we evaluated in our o1 System Card.⁠ We’re also updating o1 in ChatGPT to this version soon. The evaluations we’re sharing below reflect the performance of this new snapshot, ensuring developers have up-to-date benchmarks for this version.

o1‑2024‑12‑17 sets new state-of-the-art results on several benchmarks, improving cost-efficiency and performance.

| Category | Eval | o1-2024-12-17 | o1-preview | | --- | --- | --- | --- | | General | GPQA diamond | 75.7 | 73.3 | | MMLU (pass @1) | 91.8 | 90.8 | | Coding | SWE-bench Verified | 48.9 | 41.3 | | LiveBench (Coding) | 76.6 | 52.3 | | Math | MATH (pass @1) | 96.4 | 85.5 | | AIME 2024 (pass @1) | 79.2 | 42.0 | | MGSM (pass @1) | 89.3 | 90.8 | | Vision | MMMU (pass @1) | 77.3 | — | | MathVista (pass @1) | 71.0 | — | | Factuality | SimpleQA | 42.6 | 42.4 | | Agents | TAU-bench (retail) | 73.5 | — | | TAU-bench (airline) | 54.2 | — |

##### Model Evaluation Accuracy Across Different Metrics

Accuracy

Additionally, we have observed thato1‑2024‑12‑17 significantly outperforms gpt-4o in our function calling and Structured Outputs testing.

We are rolling out access incrementally while working to expand access to additional usage tiers and ramping up rate limits. To get started, check out the API documentation⁠.

Improvements to the Realtime API

The Realtime API⁠ enables developers to create low-latency, natural conversational experiences. It’s ideal for voice assistants, live translation tools, virtual tutors, interactive customer support systems, or even your own virtual Santa⁠. Today we’re releasing changes to address some of the most common requests from developers: a direct WebRTC integration, reduced pricing, and more control over responses.

###### WebRTC support

We’re introducing WebRTC⁠ support for the Realtime API. WebRTC is an open standard that makes it easier to build and scale real-time voice products across platforms—whether for browser-based apps, mobile clients, IoT devices, or direct server-to-server setups.

Our WebRTC integration is designed to enable smooth and responsive interactions in real-world conditions, even with variable network quality. It handles audio encoding, streaming, noise suppression, and congestion control.

With WebRTC, you can now add Realtime capabilities with just a handful of lines of Javascript:

JavaScript

1async function createRealtimeSession(localStream, remoteAudioEl, token) {2 const pc = new RTCPeerConnection();3 pc.ontrack = e => remoteAudioEl.srcObject = e.streams[0];4 pc.addTrack(localStream.getTracks()[0]);5 const offer = await pc.createOffer();6 await pc.setLocalDescription(offer);7 const headers = { Authorization: `Bearer ${token}`, 'Content-Type': 'application/sdp' };8 const opts = { method: 'POST', body: offer.sdp, headers };9 const resp = await fetch('https://api.openai.com/v1/realtime', opts);10 await pc.setRemoteDescription({ type: 'answer', sdp: await resp.text() });11 return pc;12}

Learn more about our WebRTC integration in the API documentation⁠.

New GPT-4o and GPT-4o mini realtime snapshots at lower cost

We’re releasinggpt-4o-realtime-preview-2024-12-17 as part of the Realtime API beta with improved voice quality, more reliable input (especially for dictated numbers), and reduced costs. Due to our efficiency improvements, we’re dropping the audio token price by 60% to $40/1M input tokens and $80/1M output tokens. Cached audio input costs are reduced by 87.5% to $2.50/1M input tokens.

We’re also bringing GPT‑4o mini to the Realtime API beta asgpt-4o-mini-realtime-preview-2024-12-17. GPT‑4o mini is our most cost-efficient small model and brings the same rich voice experiences to the Realtime API as GPT‑4o. GPT‑4o mini audio price is $10/1M input tokens and $20/1M output tokens. Text tokens are priced at $0.60/1M input tokens and $2.40/1M output tokens. Cached audio and text both cost $0.30/1M tokens.

These snapshots are available in the Realtime API⁠ and also in the Chat Completions API⁠ asgpt-4o-audio-preview-2024-12-17 andgpt-4o-mini-audio-preview-2024-12-17.

More control over responses

We’re shipping the following features to the Realtime API to make it easier to deliver exceptional voice-driven experiences:

• Concurrent…