AskAiml
Sign in Get started

Structured outputs — the bridge between LLMs and code

Pydantic schemas, JSON mode, and Instructor: get typed data out of a model instead of parsing strings.

💬 Module 1 7 min read Not started

Why this matters

In production, you do not want the LLM to return prose for your code to parse with regex. You want a typed object. Structured outputs are how you get that, reliably, every time. The phrase "structured outputs" appears in 90%+ of agentic AI job descriptions for a reason — it is the bridge between LLMs and traditional software.

Learning objectives

  1. Use OpenAI's Structured Outputs (Responses API and Chat Completions) with strict schemas.
  2. Use Anthropic's tool-use mode for the same outcome.
  3. Use the instructor library to do this with any provider in one line.
  4. Add Pydantic validators that auto-retry the LLM on bad output.
  5. Stream partial structured outputs to a UI.

1. Why naive JSON parsing fails

You have all done this:

python
text = llm("Return JSON: name and age for John, 30").content
import json
data = json.loads(text)  # KABOOM

Failures: extra prose around the JSON, single quotes, trailing commas, hallucinated fields, missing required fields. Production systems cannot tolerate this.

The 2026 fix: tell the model and the API to enforce a schema. Three flavors below.

python
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import Literal

client = OpenAI()

class Ticket(BaseModel):
    category: Literal["billing", "technical", "account", "other"]
    priority: Literal["low", "medium", "high"]
    summary: str = Field(..., max_length=120)

resp = client.responses.parse(
    model="gpt-4.1-mini",
    instructions="Classify this support ticket.",
    input="My card was charged twice for invoice #42.",
    text_format=Ticket,
)

t: Ticket = resp.output_parsed
print(t.category, t.priority, t.summary)

That is it. The SDK converts the Pydantic model to a JSON Schema, the API guarantees the output matches, and output_parsed is a typed Ticket.

Behind the scenes

If you want the raw schema control:

python
resp = client.responses.create(
    model="gpt-4.1-mini",
    input="...",
    text={
        "format": {
            "type": "json_schema",
            "name": "Ticket",
            "strict": True,
            "schema": Ticket.model_json_schema(),  # Pydantic gives this
        }
    }
)

Critical: for strict=True to work, every object must have:

  • additionalProperties: false
  • All properties listed in required

Pydantic does this automatically when you set model_config = ConfigDict(extra="forbid") on the model:

python
from pydantic import BaseModel, ConfigDict

class Ticket(BaseModel):
    model_config = ConfigDict(extra="forbid")
    category: str
    priority: str

3. OpenAI Structured Outputs — Chat Completions

If you are on the older Chat Completions API, use the parse helper:

python
completion = client.chat.completions.parse(
    model="gpt-4.1-mini",
    messages=[
        {"role": "system", "content": "Classify the ticket."},
        {"role": "user", "content": "My card was charged twice."},
    ],
    response_format=Ticket,
)
ticket = completion.choices[0].message.parsed

There is also a refusal field on the message — the model can refuse the request, in which case parsed is None and refusal has the explanation. Always handle both.

python
msg = completion.choices[0].message
if msg.refusal:
    handle_refusal(msg.refusal)
else:
    ticket = msg.parsed

4. Anthropic — structured outputs via tool use

Anthropic does not have a dedicated "structured output" param. Instead you define a single tool whose schema is your output, then force the model to call it.

python
import anthropic
from pydantic import BaseModel

class Invoice(BaseModel):
    vendor: str
    amount: float
    due_date: str

aclient = anthropic.Anthropic()

resp = aclient.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    tools=[{
        "name": "extract_invoice",
        "description": "Extract structured invoice data.",
        "input_schema": Invoice.model_json_schema(),
    }],
    tool_choice={"type": "tool", "name": "extract_invoice"},
    messages=[{"role": "user", "content": "Acme Corp owes us $4,250 by 2026-07-15."}],
)

tool_use = next(b for b in resp.content if b.type == "tool_use")
invoice = Invoice(**tool_use.input)
print(invoice)

tool_choice={"type":"tool","name":"..."} forces the call. Without it the model decides.

5. Instructor — provider-agnostic, the easiest path

instructor is a small library that patches OpenAI/Anthropic/Gemini/Ollama clients to take a response_model argument. It also auto-retries when Pydantic validation fails. In 2026 this is the most common pick across teams that want one wrapper.

python
# uv add instructor
import instructor
from pydantic import BaseModel, field_validator
from typing import Literal

class Order(BaseModel):
    item: str
    qty: int
    payment: Literal["card", "upi", "cod"]

    @field_validator("qty")
    @classmethod
    def positive(cls, v):
        if v <= 0:
            raise ValueError("qty must be positive")
        return v

client = instructor.from_provider("openai/gpt-4.1-mini", max_retries=3)

order = client.create(
    response_model=Order,
    messages=[{"role": "user", "content": "I want 2 shirts paid by UPI."}],
)
print(order)  # Order(item='shirt', qty=2, payment='upi')

When validation fails, instructor:

  1. Catches the ValidationError.
  2. Sends the error message back to the model.
  3. Asks it to fix the output.
  4. Tries again, up to max_retries.

That validator-driven loop catches things like negative quantities, wrong enums, regex mismatches — as if the LLM had stronger guarantees than it does.

Switch providers in one line

python
client = instructor.from_provider("anthropic/claude-haiku-4-5")
client = instructor.from_provider("google/gemini-2.5-flash")
client = instructor.from_provider("ollama/llama3.3", mode=instructor.Mode.JSON)

Same response_model API everywhere.

6. Streaming partial structured outputs

For UIs that need to show fields as they arrive (e.g. typing fields one by one), instructor and OpenAI both support streaming partials.

python
from instructor import Partial

stream = client.create(
    response_model=Partial[Order],
    messages=[{"role":"user","content":"3 books, paid card"}],
    stream=True,
)
for partial in stream:
    print(partial)  # incrementally filled Order

OpenAI Responses API has the same with client.responses.stream(...) plus text_format=Order.

7. When to use which (decision rule)

SituationPick
OpenAI only, modern appResponses API responses.parse(text_format=)
OpenAI legacy, modern appChat Completions chat.completions.parse(response_format=)
Anthropic onlyTool-use trick with forced tool_choice
Multi-providerinstructor.from_provider(...)
Local/Ollamainstructor with Mode.JSON
Need streaming partialsinstructor Partial[Model]

Default in new code: instructor. Reach for native APIs only when you need a feature instructor does not yet expose.

8. Validators are your superpower

Use Pydantic field validators to enforce semantic rules — they trigger instructor's auto-retry.

python
from pydantic import BaseModel, field_validator
import re

class CompanyEmail(BaseModel):
    address: str

    @field_validator("address")
    @classmethod
    def must_be_company(cls, v):
        if not re.match(r".*@(acme|globex)\.com$", v):
            raise ValueError("must be @acme.com or @globex.com")
        return v

If the LLM returns john@gmail.com, instructor catches the error, sends "address must be @acme.com or @globex.com" to the model, and asks again. Two-line guard, big reliability win.

Hands-on lab (3 hours)

Build extractor.py:

  1. Define a Pydantic model for BankStatementLine:
    python
    class BankStatementLine(BaseModel):
    date: date
    amount: float
    direction: Literal["credit", "debit"]
    counterparty: Optional[str]
    category: Literal["food", "transport", "rent", "salary", "other"]
  2. Read 30 lines from statement.csv (mix clean and messy lines).
  3. For each line, use instructor to extract a BankStatementLine from the raw text.
  4. Add a validator: amount must be positive; the LLM should be forced to set direction instead.
  5. On 5 deliberately broken lines (e.g. negative amount), confirm instructor retries and succeeds.
  6. Save results to statement_clean.json plus a small markdown report on retries used per line.

Acceptance criteria:

  • Average retries < 0.5 across the dataset.
  • 100% schema-valid output.
  • README explains how to swap GPT-4.1-mini for Claude Haiku 4.5 in one line.

Common pitfalls

  1. Forgetting extra="forbid" — strict schemas fail silently.
  2. Using Optional[X] everywhere — model treats them as default-None and lazily skips.
  3. Validators with side effects — keep them pure; they may run many times.
  4. Schemas with Union[A, B, C] of >5 types — model gets confused. Use a discriminated union with a type field.
  5. Asking for very large schemas in one shot — split into stages (basic fields then enrich).

Self-check

  1. What is the difference between response_format (Chat Completions) and text.format (Responses)?
  2. Why does Anthropic structured output use the tool-use trick?
  3. What does Partial[Order] give you?
  4. Why does additionalProperties: false matter for strict: true?
  5. How does instructor's auto-retry work end-to-end?

References

Sign in to save your progress and earn badges.