Hands-On: Extract Structured Data from Unstructured Text

Introduction

This is the lesson where the whole section pays off. You've learned the pieces: why structured output matters and how constrained decoding guarantees valid JSON (#1), JSON mode vs response schemas (#2), tool/function calling (#3), how to design good schemas (#4), and how to parse, validate, and retry with Pydantic (#5). Now we put them together to do one of the most common — and most valuable — jobs in AI engineering: turning messy, unstructured text into clean, typed records your code can actually use.

Support emails, invoices, resumes, contracts, chat logs, scraped pages — the world runs on unstructured text, and almost no downstream system can consume it directly. The classic fix was brittle regex and hand-written parsers. The modern fix is a few lines: define the shape you want as a schema, and let the model fill it in — reliably, because structured outputs guarantee the JSON is valid.

We'll build a real extractor end-to-end — on a messy customer-support email — and handle the parts that separate a demo from production: fields that aren't there, hallucinated values, validation, retries, batching, and the payoff of queryable data.

In this lesson you'll build:

A Pydantic schema that defines exactly what to pull out of the text
A structured-output extraction call (Claude and OpenAI) that fills it in
Guards against guessing — nulls for absent fields + an evidence trail
Validation + retries, then batch extraction over many documents
And you'll see why, once it's structured, the data is suddenly trivial to use

The Input, and the Shape You Want

Start by looking at what you're up against — a real support email is rambling, emotional, and inconsistent — and decide the exact shape you want out of it. That target shape is your schema (recall #4: the schema is a contract and a prompt). Defining it well is 80% of the job.

Here's the messy input and the SupportTicket we'll extract. Note three deliberate choices: enums for category/urgency (the model must pick from a fixed set, not invent labels), str | None for fields that might not be in the text (so the model can return null instead of guessing), and an evidence field that forces the model to quote the source — our grounding trick, more on that below.

from enum import Enum
from pydantic import BaseModel, Field

# The messy, real-world input
EMAIL = """From: jane.doe@example.com
Subject: still no refund??

Hi - I returned order A-4471 over THREE WEEKS ago and still haven't seen the
refund on my card. This is the second time I'm writing. I need this fixed today
or I'll dispute the charge with my bank. - Jane"""

# The target shape = your CONTRACT for the extraction
class Category(str, Enum):
    billing = "billing"
    shipping = "shipping"
    technical = "technical"
    other = "other"

class Urgency(str, Enum):
    low = "low"
    medium = "medium"
    high = "high"

class SupportTicket(BaseModel):
    category: Category
    urgency: Urgency
    customer_name: str | None = Field(description="Sender's name, or null if not stated")
    order_id: str | None = Field(description="e.g. 'A-4471', or null if absent")
    summary: str = Field(description="One-sentence summary of the issue")
    requested_action: str = Field(description="What the customer wants done")
    evidence: str = Field(description="A short quote from the email backing the category & urgency")

An infographic titled 'Extract Structured Data from Unstructured Text'. On the left, a messy raw customer support email from Jane about a missing refund for order A-4471, threatening a chargeback. A bold arrow passes through a central box labelled 'LLM + Pydantic schema (SupportTicket) + structured outputs'. On the right, a clean typed JSON record with fields: category = billing, urgency = high, customer_name = Jane, order_id = A-4471, summary, requested_action, and an evidence quote pulled from the email. An amber note says fields not present in the text become null and the model must not guess. A bottom banner reads: messy text in, typed validated records out; valid does not mean correct, so still spot-check.

That's the entire game in one picture: unstructured text in → a typed, validated record out. Everything below is making that transform trustworthy.

The Extraction Call

Now the call itself. We hand the model the email and the schema; structured outputs guarantees the response parses into a SupportTicket. With the parse helper you get a typed object back directly — no json.loads, no try/except around parsing (that guarantee is exactly what #2 bought us).

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

def extract(email: str) -> SupportTicket:
    resp = client.beta.messages.parse(       # structured-output parse helper (beta)
        model="claude-sonnet-4-6",
        betas=["structured-outputs-2025-11-13"],   # structured outputs is a beta
        max_tokens=512,
        messages=[{
            "role": "user",
            "content": "Extract a support ticket from this email. "
                       "Use null for anything not stated - do NOT guess.\n\n" + email,
        }],
        output_format=SupportTicket,         # <- your Pydantic schema = the contract
    )
    return resp.parsed_output                # <- already a validated SupportTicket

t = extract(EMAIL)
print(t.category, t.urgency, t.order_id, t.customer_name)
# -> Category.billing Urgency.high A-4471 Jane

Each field is now first-class data: t.urgency is Urgency.high, t.order_id == "A-4471". The same extractor in OpenAI is identical in spirit — same Pydantic schema, different parse helper:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY

def extract(email: str) -> SupportTicket:
    resp = client.responses.parse(
        model="gpt-5.5",
        input=[{"role": "user",
                "content": "Extract a support ticket. Use null for anything not stated.\n\n" + email}],
        text_format=SupportTicket,           # <- the SAME Pydantic schema
    )
    return resp.output_parsed

One schema, either provider. The schema is the contract; the SDK just enforces it.

Don't Let It Guess: Grounding the Extraction

Here's where most extraction tutorials stop — and where real systems break. Structured outputs guarantees the JSON is valid, not that it's correct (the hard-won lesson from #1). The model will happily invent a plausible order_id or a customer name that isn't in the email, and hand it back in perfectly-typed JSON. Garbage that passes the schema is still garbage.

Two cheap, high-leverage defenses, both already in our schema:

Make absent fields nullable + tell the model to use null. order_id: str | None plus "use null for anything not stated — do NOT guess" turns "make something up" into an honest "not present." (Structured outputs require every field to be present, but a field can be null — that's how you model "optional" correctly.)
Demand evidence. The evidence field forces the model to quote the text that justifies its category/urgency. This grounds the extraction (you can check the quote really appears in the source) and, as a bonus, tends to make the classification itself more accurate — it's chain-of-thought, aimed at a field.

Treat extraction like a deposition: every claim must be backed by something in the document, and "I don't know" is a valid, preferred answer over a confident fabrication.

Validate, Then Retry

Schema-valid isn't the same as business-valid (the second kind of "wrong" from #5). Maybe order_id must match ^[A-Z]-\d{4}$, or a high-urgency ticket must carry evidence. Those are business rules, and Pydantic validators enforce them — turning a silent bad record into a loud error you can act on:

from pydantic import model_validator

# Drop a business rule into SupportTicket. The retry loop from the last lesson
# (or instructor's max_retries) turns this error back into a prompt for the model.
@model_validator(mode="after")
def high_urgency_needs_evidence(self):
    if self.urgency is Urgency.high and not self.evidence.strip():
        raise ValueError("high-urgency tickets must cite evidence from the email")
    return self

When that validator fires, you do exactly what the previous lesson taught: feed the error back to the model and retry. Roll your own loop, or let instructor's max_retries do it (we'll use it in a moment). The principle holds: validate before anything downstream trusts the data — extraction is only as good as the checks behind it.

Scale It: Batch Extraction

One email is a demo; a real pipeline runs over thousands. The pattern is a loop with per-item error handling so a single malformed document can't take down the whole run:

emails = [EMAIL, email_2, email_3, ...]   # a folder, a DB query, a CSV column...

tickets, failures = [], []
for e in emails:
    try:
        tickets.append(extract(e))
    except Exception as err:               # one bad email must NOT kill the whole batch
        failures.append((e[:40], str(err)))

print(f"{len(tickets)} extracted, {len(failures)} failed")

That's the production skeleton: extract, isolate failures, keep going. From here you'd log the failures for review, retry them, and write the good tickets to a database, a CSV, or a dataframe. Note the cost shape — this is one model call per document, so batching is also where you think about concurrency, rate limits, and spend (all of which we tackle in the Production container).

The Payoff: Now It's Just Data

Why go to all this trouble? Because the moment the text is structured, it stops being an AI problem and becomes a trivial one. Counting, filtering, routing, charting — all the things you couldn't do with a wall of prose — are now three lines of ordinary Python. Run this (no API needed) to feel the difference:

from collections import Counter

# After extraction, the data is just... data. Query it with plain Python:
tickets = [
    {"category": "billing",   "urgency": "high"},
    {"category": "shipping",  "urgency": "low"},
    {"category": "billing",   "urgency": "high"},
    {"category": "technical", "urgency": "medium"},
]

by_category = Counter(t["category"] for t in tickets)
high_priority = [t for t in tickets if t["urgency"] == "high"]

print(by_category)                                  # Counter({'billing': 2, 'shipping': 1, 'technical': 1})
print(len(high_priority), "tickets need urgent attention")   # 2 tickets need urgent attention

That's the whole point of extraction: it turns language into something a spreadsheet, a dashboard, a router, or a database can consume. The LLM did the messy part once; everything after is deterministic code you can test and trust.

The Shortcut: instructor

You've now built extraction from first principles — schema, call, grounding, validation, retries, batching — so you understand every moving part. In day-to-day work, the instructor library packages all of it behind one call: pass a response_model and a max_retries, and it handles the structured-output plumbing and the validate-and-retry loop for you, across providers:

import instructor

# Provider-agnostic, with validation + retries built in (from the previous lesson):
client = instructor.from_provider("anthropic/claude-sonnet-4-6")  # or "openai/gpt-5.5"

ticket = client.chat.completions.create(
    response_model=SupportTicket,   # validate against the schema...
    max_retries=3,                  # ...and re-ask the model on a validation failure
    messages=[{"role": "user", "content": "Extract a support ticket.\n\n" + EMAIL}],
)
print(ticket.category)

Same result, less boilerplate. Reach for it once you know what it's doing under the hood — which now you do. (This is the course's pattern everywhere: build it raw to understand it, then use the library to ship it.)

Pitfalls & Common Mistakes

Trusting valid JSON as correct. Structured outputs stop malformed output, not wrong output. Always spot-check, and ground with nulls + evidence. Measure accuracy with evals (Container 4).
Forcing a value for absent fields. Non-nullable fields make the model invent data. Use str | None and instruct it to return null.
Over-stuffed schemas. Twenty fields in one call hurts accuracy. Extract what you need; for very rich documents, split into a few focused extractions (prompt chaining, last section).
No business-rule validation. Schema types catch shape; they don't catch "quantity = -3" or a malformed ID. Add validators.
Batch with no error isolation. One bad document shouldn't crash the run — wrap each item and collect failures.
Documents longer than the context window. You can't extract from what doesn't fit. Chunk long docs first (we cover chunking in the RAG container) and extract per chunk.

🧪 Try It Yourself

Take the SupportTicket schema and add one field: mentioned_competitor: str | None (the name of a rival the customer threatens to switch to, or null). Run the extractor on two emails — one that names a competitor, one that doesn't — and confirm you get the name in the first and a clean null in the second (not a hallucinated guess).

Then add a model_validator rule of your own — say, a billing ticket must include an order_id — and watch it reject an email that's missing one. Which mattered more for correctness here: the schema, or the instruction to use null and cite evidence?

Extraction Studio — extract a SupportTicket from a messy customer email and toggle the two grounding defenses (make absent fields nullable and instruct the model to use null, and add an evidence field that quotes the source). Pick the refund email (no competitor mentioned) or the crash email (no order id or name). Without grounding, the schema-valid record hallucinates plausible, perfectly-typed values for the fields that are not in the text — a fabricated competitor, a made-up order id — and the verdict flags them; with grounding on, those fields come back as an honest null and the evidence field quotes the source. Lands the finale: structured outputs guarantee the JSON is valid, never that it is correct, so valid ≠ correct and you must ground the extraction.

Key Takeaways

Extraction = define the shape, then let the model fill it. A Pydantic schema is the contract; structured outputs guarantee the JSON is valid.
Valid ≠ correct. Guard against hallucinated fields with nullable types + "use null, don't guess" and an evidence field that quotes the source.
Validate business rules with Pydantic validators, and retry by feeding the error back (or let instructor do it).
Batch with per-item error handling so one bad document doesn't sink the run.
Once text is structured, it's just data — countable, filterable, routable with plain code. That's the payoff.
Build it raw to understand it; reach for instructor to ship it.

Next: we open a new section — Context Engineering — the 2026 shift from crafting a single prompt to assembling the entire context (instructions, memory, retrieved data, tools, state) the model sees.