JSON Mode & Response Schemas

Last lesson made the case: prompt-and-pray JSON is unreliable, and the real fix is constrained decoding. This lesson is the hands-on first rung — how to actually request guaranteed-shape output from the API.

There are two levels of guarantee, and confusing them is the most common beginner mistake: JSON mode (you get valid JSON) and response schemas / structured outputs (you get exactly your schema). They sound similar; they're worlds apart in reliability. We'll see both, the code to use them, and the clean idiomatic way: define your schema as a Pydantic/Zod object and get a typed result back.

You'll learn:

• JSON mode vs response schemas — and why the difference matters
• How to request structured output (OpenAI, Anthropic, Gemini)
• The clean pattern: Pydantic/Zod model → typed object back
• Schema basics that help the model comply

Both make output more structured — but they guarantee different things:

• JSON mode guarantees the output is syntactically valid JSON — it'll parse. But it does not enforce your structure: fields can be missing, types can be wrong (age as a string), extra fields can appear. You still get ~2–5% schema mismatch.
• Response schemas (structured outputs, strict) guarantee the output matches your exact schema — the right fields, the right types, only the allowed enum values. Achieved by constraining generation to a grammar built from your schema → ~100% schema compliance (< 0.1% fail).

In short: JSON mode promises valid; response schemas promise valid and correctly shaped. For anything you build a pipeline on, you want the second.

JSON mode tells the model 'your entire output must be a single valid JSON value.' It kills the syntax failures from last lesson — no preamble, no markdown fences, no trailing commas — so JSON.parse() won't throw.

But it's only half the battle: JSON mode doesn't know or enforce your fields. The model might return {"summary": "..."} and forget priority entirely, or return priority as "high" instead of your "P0" enum. It's valid JSON that's wrong for your code.

# OpenAI — JSON mode (valid JSON, but NOT your schema)
resp = client.chat.completions.create(
    model="gpt-5.5",
    response_format={"type": "json_object"},   # ← just 'valid JSON'
    messages=[{"role": "user", "content": "... return JSON ..."}],
)

Use JSON mode only when you genuinely just need some parseable JSON and will validate the shape yourself. For production, climb one more rung.

Response schemas (a.k.a. structured outputs, strict mode) are the real deal. You hand the API a JSON Schema describing your exact shape, and the provider constrains generation to it — every token must keep the output on a valid path through your schema. The result conforms 100%: right fields, right types, only your allowed values.

# OpenAI — structured outputs (strict): matches YOUR schema exactly
resp = client.chat.completions.create(
    model="gpt-5.5",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "ticket",
            "strict": True,                       # ← the magic flag
            "schema": {
                "type": "object",
                "properties": {
                    "category": {"enum": ["billing", "technical", "account"]},
                    "priority": {"enum": ["P0", "P1", "P2"]},
                    "summary":  {"type": "string"},
                },
                "required": ["category", "priority", "summary"],
                "additionalProperties": False,
            },
        },
    },
    messages=[{"role": "user", "content": "Triage: 'charged twice, can't log in!'"}],
)

That strict: True is what flips on constrained decoding. The output is guaranteed to be a {category, priority, summary} object with valid enums — no parsing roulette.

All three majors support it in 2026 — same idea, different knob:

• OpenAI: response_format={"type": "json_schema", "json_schema": {…, "strict": True}}.
• Anthropic (Claude): structured outputs (a 2025 beta) via an output_config.format of type: "json_schema" (with the structured-outputs beta header); Claude also offers strict tool use for tool arguments.
• Gemini: response_mime_type="application/json" plus a response_schema.

The concept is identical — give the model a schema, get schema-valid output. Only the parameter names differ, so wrap it behind your provider abstraction (from the SDK lesson) and you can swap freely.

Hand-writing JSON Schema is verbose and error-prone. The idiomatic pattern is to define your shape as a typed model and let the SDK do the rest:

from pydantic import BaseModel
from typing import Literal

class Ticket(BaseModel):
    category: Literal["billing", "technical", "account"]
    priority: Literal["P0", "P1", "P2"]
    summary: str

resp = client.chat.completions.parse(        # note: .parse, not .create
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Triage: 'charged twice, can't log in!'"}],
    response_format=Ticket,                  # ← pass the model itself
)
ticket = resp.choices[0].message.parsed       # ← a typed Ticket object
print(ticket.priority)                         # "P0"  (guaranteed valid)

The SDK converts your model to JSON Schema, sends it with strict, constrains the generation, and deserializes the response back into a Ticket instance. You write a class, you get a typed object — no manual schema, no json.loads, no shape-checking. (JavaScript/TypeScript: the same with Zod.) This is how structured output should feel.

Even with constrained decoding, how you write the schema shapes quality:

• Use enums for fixed choices. Literal["P0","P1","P2"] beats a free string — it makes the value checkable, not just the type.
• Name fields clearly and add descriptions — the model reads them to decide what goes where (constrained decoding fixes the shape, not the meaning).
• Mark required fields and set additionalProperties: false so nothing unexpected sneaks in.
• Keep it as flat and small as the task allows — deeply nested or huge schemas are harder for the model to fill correctly (shape is guaranteed; correctness isn't).

(We go deep on schema design in the next-next lesson — it matters even more for tool schemas.)

Spot the right rung. For each task, decide: does JSON mode suffice, or do you need a strict response schema?

1. Quick script: dump a model's freeform analysis into a file as parseable JSON, you'll eyeball it. → ?
2. Production router: classify a ticket into billing | technical | account and your code branches on it. → ?
3. Pipeline step: extract {name, age:int, email} to insert into a typed database row. → ?

Answers: 1 → JSON mode is fine (you just need some valid JSON). 2 and 3 → strict response schema (the value must be a known enum / the type must be a real int — your code depends on it). Rule of thumb: the moment another piece of code consumes the output, use a strict schema. Bonus: write the Pydantic model for task 3 — three lines, and you're done.

• "JSON mode gives me my schema." No — it gives valid JSON; fields can still be missing or mistyped. Response schemas give your shape.
• "Structured outputs are just JSON mode renamed." No — strict mode adds constrained decoding to a grammar built from your schema (100% compliance).
• "I have to hand-write JSON Schema." No — define a Pydantic/Zod model; the SDK converts it and hands you a typed object.
• "With a schema, the data is correct." Still no — schema = shape, not truth (last lesson). Validate values separately.
• "Each provider is totally different." Same concept (give a schema), different parameter — abstract it and swap.

• JSON mode = valid JSON (no syntax errors) but not your schema (2–5% mismatch). Response schemas / structured outputs (strict) = your exact shape via constrained decoding (~100% compliance) — the production default.
• Request it with response_format={type:'json_schema', …, strict:true} (OpenAI), output_config.format (Anthropic), or response_schema (Gemini) — same idea, different knob.
• The clean pattern: define a Pydantic/Zod model → SDK converts it, constrains generation, and returns a typed object — no manual schema or parsing.
• Write schemas well: enums for fixed choices, clear names + descriptions, required fields, additionalProperties:false.
• Remember: schema guarantees shape, not truth — still validate the content.

Next: the same machinery powers something bigger — function / tool calling, where the model returns structured arguments your code executes. That's the gateway to tools and agents.