Constrained Generation With Reasoning

From text to task: Constrained generation for structured extraction in R1

GLM 5.2 is live! Opus-level intelligence at open-source rates. Pay per token on serverless. Try it today.

Blog

Constrained Generation With Reasoning From text to task: Constrained generation for structured extraction in R1

PUBLISHED 2/1/2025

Table of Contents What we’ll cover How it works Output

The response from R1

The reasoning

The parsed JSON data

Output

The response from R1

The reasoning

The parsed JSON data Additional resources

Table of Contents

What is constrained generation and why is it useful

Constrained generation is a technique in natural language processing (NLP) where language models are guided to produce text that adheres to specific predefined rules or structures. This approach is particularly useful in applications requiring structured outputs, such as generating code, creating formatted documents, or producing data in formats like JSON. By enforcing constraints during the text generation process, models can ensure outputs that are not only coherent but also conform to the desired structure, enhancing both the utility and reliability of the generated content. What we’ll cover

• How constrained generation works • Guiding model token selection • Constrained decoding for structured outputs

• Reasoning models and structured extraction • The role of constrained generation in reasoning models • Fireworks' JSON mode for reasoning models

• Examples of constrained generation in action • Structured Q&A with reasoning • Healthcare records with AI-driven summaries • Computer system specifications with structured recommendations

• Conclusion • Why structured generation improves AI reliability • Future applications and best practices

How does it work

The process of constrained generation involves manipulating a model's token generation to restrict its next-token predictions to only those that do not violate the required output structure. This can be achieved through various methods, such as constrained decoding, where the model's output is directed to follow specific patterns or formats. For instance, in structured generation tasks, constrained decoding can simplify the next-token prediction space, accelerating generation by allowing some token generation steps to be skipped. Additionally, by focusing only on generating the necessary parts of the output and bypassing boilerplate sections, the overall efficiency of the generation process is improved.

Implementing constrained generation not only enhances the quality of the output by ensuring adherence to desired formats but may also improve performance. By reducing the complexity of the generation task and narrowing down the prediction space, the model can generate outputs more quickly and with greater accuracy. This efficiency gain is particularly beneficial in applications where rapid and reliable generation of structured text is crucial. Constrained generation in reasoning models

In the context of reasoning models, such as the recently released DeepSeek R1 , constrained generation plays a pivotal role in ensuring that outputs adhere to specific formats and structures. The DeepSeek R1 model exemplifies this by incorporating a unique mechanism: it generates a reasoning process enclosed within and tokens, followed by a JSON-formatted output. This structured approach allows the model to transparently display its thought process before presenting the final result. Notably, the JSON schema applies exclusively to the JSON section that follows the tags, ensuring that the reasoning process and the final output are clearly delineated and properly formatted. The caller can employ simple output parsing to separate the reasoning section from the structured output. Example 1: Simple Q&A

In this section, we'll demonstrate how to utilize the DeepSeek R1 reasoning model in JSON mode using the Fireworks API . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56

Import necessary libraries

import json import re from pydantic import BaseModel from openai import OpenAI import os

Initialize the Fireworks client

client = OpenAI ( base_url = "https://api.fireworks.ai/inference/v1" , api_key = os . getenv ( "FIREWORKS_API_KEY" ) , )

Define the output schema using Pydantic

class QAResult ( BaseModel ) : question : str answer : str

Prepare the user input

user_input = "Who wrote 'Pride and Prejudice'?"

Construct the messages payload

messages = [ { "role" : "user" , "content" : user_input } ]

Make the API call to DeepSeek R1

response = client . chat . completions . create ( model = "accounts/fireworks/models/deepseek-r1" , messages = messages , response_format = { "type" : "json_object" , "schema" : QAResult . model_json_schema ( ) } , max_tokens = 1000 , # Adjust as needed to prevent truncation )

Extract the content of the response

response_content = response . choices [ 0 ] . message . content print ( f"Response content: { response_content } " )

Use regular expressions to extract the reasoning and JSON parts.

The reasoning is enclosed within ... tags,

and the JSON part follows the tag.

reasoning_match = re . search ( r" (.*?) " , response_content , re . DOTALL ) json_match = re . search ( r" \s*(\{.*\})" , response_content , re . DOTALL )

Extract reasoning

reasoning = reasoning_match . group ( 1 ) . strip ( )

Extract JSON string

json_str = json_match . group ( 1 ) . strip ( )

Directly parse the JSON string into a Pydantic model

qa_result = QAResult . model_validate_json ( json_str )

Output the extracted reasoning and the parsed Pydantic model

print ( f"\nReasoning: { reasoning } " ) print ( f"\nQAResult: { qa_result } " )

How it works

User Input: A question is submitted to the model (e.g., "Who wrote Pride and Prejudice?"). Constrained Generation: The model first produces its reasoning, enclosed in ... , ensuring a structured explanation before providing the answer. Schema Enforcement: The JSON-formatted response follows a Pydantic-defined schema , ensuring structured data. Parsing and Validation: The reasoning section and JSON output are extracted separately, maintaining cleanly structured and machine-readable responses .

Let’s show another example, this time for a health care use case. Example 2: Structured healthcare data generation with reasoning JSON mode

This example...