JSON Mode and Beyond

In this tutorial, you'll learn how to:

• Guarantee structured output using our chat completions API
• This can be done using JSON Schema / Pydantic models, (schemaless) JSON Mode, regular expressions, and multiple choice.

One of the simplest forms of structured output is multiple choice.

The only possible outputs of the following code snippet are positive or negative.

Note

Make sure you have set the TELNYX_API_KEY environment variable

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("TELNYX_API_KEY"),
    base_url="https://api.telnyx.com/v2/ai",
)

chat_completion = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "Classify the sentiment of this review as positive or negative."},
        {"role": "user", "content": "The staff went above and beyond! I had a great stay."}
    ],
    temperature=0.0,
    extra_body={
      "guided_choice": ["positive", "negative"]
    }
)
print(chat_completion.choices[0].message.content)

Now let's say we want to capture an explanation for the classification as well. We can leverage the guided_json field for this.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("TELNYX_API_KEY"),
    base_url="https://api.telnyx.com/v2/ai",
)

chat_completion = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "First describe the sentiment of the following review and then use that to classify the sentiment as positive or negative."},
        {"role": "user", "content": "The staff went above and beyond! I had a great stay."}
    ],
    temperature=0.0,
    extra_body={
      "guided_json": {
        "type": "object",
        "properties": {
          "sentiment": {
            "type": "string",
            "enum": ["positive", "negative"]
          },
          "explanation": {"type": "string"}
        },
        "required": ["sentiment", "explanation"]
      }
    }
)
print(chat_completion.choices[0].message.content)

This will ensure a JSON response with the same schema as the following

{
  "sentiment": "positive",
  "explanation": "The review mentions that the staff went above and beyond and the user had a great stay, which indicates a positive sentiment."
}

The above is helpful to see the raw JSON Schema specification being sent via API. However, for practical purposes, using Pydantic models can help simplify generating the specs. The following is functionally equivalent to the previous example.

import os
from enum import Enum
from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    api_key=os.getenv("TELNYX_API_KEY"),
    base_url="https://api.telnyx.com/v2/ai",
)

class Sentiment(str, Enum):
    positive = 'positive'
    negative = 'negative'

class SentimentAnalysis(BaseModel):
    explanation: str
    sentiment: Sentiment

chat_completion = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "First describe the sentiment of the following review and then use that to classify the sentiment as positive or negative."},
        {"role": "user", "content": "The staff went above and beyond! I had a great stay."}
    ],
    temperature=0.0,
    extra_body={
      "guided_json": SentimentAnalysis.model_json_schema()
    }
)
print(chat_completion.choices[0].message.content)

We also support the schema-less JSON mode provided by OpenAI

import os
from openai import OpenAI


client = OpenAI(
    api_key=os.getenv("TELNYX_API_KEY"),
    base_url="https://api.telnyx.com/v2/ai",
)

chat_completion = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "First describe the sentiment of the following review and then use that to classify the sentiment as positive or negative. Please respond using a JSON object."},
        {"role": "user", "content": "The staff went above and beyond! I had a great stay."}
    ],
    temperature=0.0,
    response_format={"type": "json_object"}
)
print(chat_completion.choices[0].message.content)

You can do a lot with regular expressions, and now you can ensure language model outputs follow a regex. Something we see a lot, especially in a voice context, is wanting to limit the length of a response. Below is a toy example limiting a response to a sentence with 20 words or fewer.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("TELNYX_API_KEY"),
    base_url="https://api.telnyx.com/v2/ai",
)

chat_completion = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a concise and helpful AI assistant."},
        {"role": "user", "content": "Can you tell me about 10DLC?"}
    ],
    temperature=0.0,
    extra_body={
      "guided_regex": "(\\w+\\s*){1,20}[\.!?]"
    }
)
print(chat_completion.choices[0].message.content)