Introducing Structured Outputs in the API

Introducing Structured Outputs in the API | OpenAI

August 6, 2024

Introducing Structured Outputs in the API

We are introducing Structured Outputs in the API—model outputs now reliably adhere to developer-supplied JSON Schemas.

Share

Last year at DevDay, we introduced JSON mode—a useful building block for developers looking to build reliable applications with our models. While JSON mode improves model reliability for generating valid JSON outputs, it does not guarantee that the model’s response will conform to a particular schema. Today we’re introducing Structured Outputs in the API, a new feature designed to ensure model-generated outputs will exactly match JSON Schemas provided by developers.

Generating structured data from unstructured inputs is one of the core use cases for AI in today’s applications. Developers use the OpenAI API to build powerful assistants that have the ability to fetch data and answer questions via function calling⁠, extract structured data for data entry, and build multi-step agentic workflows that allow LLMs to take actions. Developers have long been working around the limitations of LLMs in this area via open source tooling, prompting, and retrying requests repeatedly to ensure that model outputs match the formats needed to interoperate with their systems. Structured Outputs solves this problem by constraining OpenAI models to match developer-supplied schemas and by training our models to better understand complicated schemas.

On our evals of complex JSON schema following, our new modelgpt-4o-2024-08-06 with Structured Outputs scores a perfect 100%. In comparison,gpt-4-0613 scores less than 40%.

With Structured Outputs,gpt-4o-2024-08-06 achieves 100% reliability in our evals, perfectly matching the output schemas.

How to use Structured Outputs

We’re introducing Structured Outputs in two forms in the API:

1. Function calling: Structured Outputs viatools is available by settingstrict: true within your function definition. This feature works with all models that support tools, including all modelsgpt-4-0613 andgpt-3.5-turbo-0613 and later. When Structured Outputs are enabled, model outputs will match the supplied tool definition.

JSON

1POST /v1/chat/completions2{3 "model": "gpt-4o-2024-08-06",4 "messages": [5 {6 "role": "system",7 "content": "You are a helpful assistant. The current date is August 6, 2024. You help users query for the data they are looking for by calling the query function."8 },9 {10 "role": "user",11 "content": "look up all my orders in may of last year that were fulfilled but not delivered on time"12 }13 ],14 "tools": [15 {16 "type": "function",17 "function": {18 "name": "query",19 "description": "Execute a query.",20 "strict": true,21 "parameters": {22 "type": "object",23 "properties": {24 "table_name": {25 "type": "string",26 "enum": ["orders"]27 },28 "columns": {29 "type": "array",30 "items": {31 "type": "string",32 "enum": [33 "id",34 "status",35 "expected_delivery_date",36 "delivered_at",37 "shipped_at",38 "ordered_at",39 "canceled_at"40 ]41 }42 },43 "conditions": {44 "type": "array",45 "items": {46 "type": "object",47 "properties": {48 "column": {49 "type": "string"50 },51 "operator": {52 "type": "string",53 "enum": ["=", ">", "=", ""29 lt = "="32 ne = "!="333435class OrderBy(str, Enum):36 asc = "asc"37 desc = "desc"383940class DynamicValue(BaseModel):41 column_name: str424344class Condition(BaseModel):45 column: str46 operator: Operator47 value: Union[str, int, DynamicValue]484950class Query(BaseModel):51 table_name: Table52 columns: list[Column]53 conditions: list[Condition]54 order_by: OrderBy555657client = OpenAI()5859completion = client.beta.chat.completions.parse(60 model="gpt-4o-2024-08-06",61 messages=[62 {63 "role": "system",64 "content": "You are a helpful assistant. The current date is August 6, 2024. You help users query for the data they are looking for by calling the query function.",65 },66 {67 "role": "user",68 "content": "look up all my orders in may of last year that were fulfilled but not delivered on time",69 },70 ],71 tools=[72 openai.pydantic_function_tool(Query),73 ],74)7576print(completion.choices[0].message.tool_calls[0].function.parsed_arguments)

Native Structured Outputs support is also available forresponse_format.

Python

1from pydantic import BaseModel23from openai import OpenAI456class Step(BaseModel):7 explanation: str8 output: str91011class MathResponse(BaseModel):12 steps: list[Step]13 final_answer: str141516client = OpenAI()1718completion = client.beta.chat.completions.parse(19 model="gpt-4o-2024-08-06",20 messages=[21 {"role": "system", "content": "You are a helpful math tutor."},22 {"role": "user", "content": "solve 8x + 31 = 2"},23 ],24 response_format=MathResponse,25)2627message = completion.choices[0].message28if message.parsed:29 print(message.parsed.steps)30 print(message.parsed.final_answer)31else:32 print(message.refusal)

Additional use cases

Developers frequently use OpenAI’s models to generate structured data for various use cases. Some additional examples include:

Dynamically generating user interfaces based on the user’s intent

For example, developers can use Structured Outputs to create code- or UI-generating applications. All of the following examples use the sameresponse_format, and can be used to generate varying UIs based on user input.

System

You are a user interface assistant. Your job is to help users visualize their website and app ideas.

Response format

View JSON schema

Assistant

1{2 "type": "div",3 "label": "",4 "children": [5 {6 "type": "header",7 "label": "",8 "children": [9 {10 "type": "div",11 "label": "Green Thumb Gardening",12 "children": [],13 "attributes": [{ "name": "className", "value": "site-title" }]14 },15 {16 "type": "div",17 "label": "Bringing Life to Your Garden",18 "children": [],19 "attributes": [{ "name": "className", "value": "site-tagline" }]20 }21 ],22 "attributes": [{ "name": "className", "value": "header" }]23 },24 {25 "type": "section",26 "label": "",27 "children": [28 {29 "type": "div",30 "label": "",31 "children": [32 {33 "type": "div",34 "label": "About Us",35 "children": [36 {37 "type": "div",38 "label": "At Green Thumb Gardening, we specialize in transforming your outdoor spaces into beautiful, thriving gardens. Our team has decades of experience in horticulture and landscape design.",39 "children": [],40 "attributes": [41 { "name": "className", "value":…