Prompt Chaining

Introduction

Welcome to the first and most fundamental of the five workflow patterns (L115). Prompt chaining is the simplest way to use more than one LLM call — and it's the foundation the others build on. The idea in one line: don't ask one prompt to do many hard things; decompose the task into a sequence of easy steps, where each step processes the output of the last.

Why start here? Because the instinct of every beginner is to write one giant mega-prompt — "read this email, decide the refund, write a reply, and translate it" — and then wonder why the output is inconsistent. Chaining is the fix, and it's a mindset shift: from prompting to engineering a pipeline of focused, verifiable steps.

In this lesson:

Why decomposing beats one mega-prompt (the accuracy argument, with numbers)
Building a chain from scratch — it's just functions passing outputs along
The gate — the programmatic check between steps that makes a chain reliable
The honest tradeoff (latency, error propagation), and when not to chain
The framework (LCEL) — once you've built it by hand

An infographic titled 'Prompt Chaining — decompose into checkable steps'. Don't ask one prompt to do five hard things; decompose the task into a sequence of easy, focused steps and put a validation gate between them. THE CHAIN: input (a messy email) → step 1 extract facts → gate → step 2 draft reply → gate → step 3 translate → output (a polished reply); each step processes the previous step's output. Why decompose: each step is an easier, focused task the model does more reliably and that you can verify; decomposition lifts accuracy roughly 20 to 50 percent on hard tasks and can turn a 50%-reliable feature into 95%; roots are least-to-most prompting (Zhou 2022). The gate (not optional): a programmatic check between steps that validates the handoff (schema, policy, quality) and on failure can retry, escalate, or halt; gates are reliability engineering — they catch an error before it cascades downstream. The honest tradeoff: you trade latency and total cost (more sequential calls) for reliability and control, and in a chain an early error amplifies because every later step builds on it, so without gates one bad step ships garbage. Prompt chaining is not chain-of-thought: CoT is reasoning inside one prompt, prompt chaining is multiple separate calls you can log, inspect, and branch between. Use it when a task splits into fixed ordered subtasks with linear dependencies — pipelines, draft then review then refine (self-correction), extract then transform then format. Banner: decompose one hard prompt into a chain of easy ones and gate every handoff; each step gets simpler and verifiable, and a gate stops an early mistake from poisoning everything after it.

Why Decompose? One Job per Step

Here's the core insight, and it's backed by research: a model is far more reliable at a small, focused task than at a big, multi-part one. Break the work into steps and two things improve at once — accuracy (each step is easier) and control (you can inspect and validate each step's output).

The numbers are striking. Decomposing a hard task into focused steps improves accuracy by ~20–50% (Google / ACL 2024); in practical terms it can turn a "50%-reliable" AI feature into a "95%-reliable" one. On research-style tasks, multi-step chains hit ~89% success vs ~67% for a single mega-prompt; on analysis tasks, ~91% vs ~71%.

This isn't a new trick — it's the LLM version of a deep idea in problem-solving: decomposition. The academic roots are Least-to-Most prompting (Zhou et al., 2022) — break a complex problem into ordered subproblems and solve them in sequence, each using the answers from the last — and Decomposed Prompting (Khot et al., 2022). Least-to-Most hit 99.7% on the compositional SCAN benchmark, beating chain-of-thought by 80+ points. Decomposition is one of the highest-leverage moves in all of prompt engineering.

The mental shift: stop thinking "how do I write the perfect prompt?" and start thinking "what's the sequence of simple prompts that gets me there — and where can I check the work?"

Build a Chain From Scratch

A prompt chain isn't a framework feature — it's ordinary code. You call the LLM, take its output, and feed it into the next call. That's the whole thing:

import anthropic
client = anthropic.Anthropic()

def call(prompt: str) -> str:
    return client.messages.create(
        model="claude-opus-4-8", max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    ).content[0].text

# A 3-step CHAIN — each step's OUTPUT becomes the next step's INPUT.
def handle_email(email: str) -> str:
    facts = call(f"Extract intent, order id, and refund amount as JSON:\n{email}")
    draft = call(f"Write a refund reply using ONLY these facts:\n{facts}")
    reply = call(f"Translate this to the customer's language:\n{draft}")
    return reply
# Three EASY prompts instead of one impossible "read this email, decide the
# refund, write the reply, and translate it — all at once" mega-prompt.

Read the three lines of handle_email: each call(...) is a focused, easy prompt, and the output of one is literally the input to the next (facts → draft → reply). Compare that to the one mega-prompt it replaces — "read this email, decide the refund, write the reply, and translate it, all at once" — which forces the model to juggle four jobs and get them all right in a single shot.

One detail that matters more than it looks: the handoffs. Each step should produce output in a structured, predictable shape (JSON, a list, XML tags) so the next step can consume it reliably. "Extract the fields as JSON" isn't decoration — structured handoffs are what keep the chain from breaking on step 2's free-form prose. (For real pipelines, validate the JSON with Pydantic or structured outputs.)

The Gate — What Makes a Chain Reliable

Now the part that separates a toy chain from a production one. A bare chain has a hidden danger: errors propagate and amplify. If step 1 extracts the wrong refund amount, step 2 drafts a reply around that wrong amount, and step 3 faithfully translates the mistake — the error compounds down the chain, and you ship garbage. (Unlike a parallel workflow, where errors stay isolated, a chain infects everything downstream.)

The fix is a gate: a programmatic check between steps that validates the handoff before continuing.

# A GATE = a programmatic check BETWEEN steps. It turns the chain from
# "hope each step is right" into "verify each handoff before continuing."
def handle_email(email: str) -> str:
    facts = call(EXTRACT.format(email=email))
    if not valid_json(facts):                       # 🚦 gate 1: structural check (pure code)
        facts = call(EXTRACT.format(email=email))    #    retry once

    draft = call(DRAFT.format(facts=facts))
    if not amount_within_policy(draft, facts):      # 🚦 gate 2: policy check
        return escalate_to_human(email, draft)       #    HALT — don't ship a bad refund

    return call(TRANSLATE.format(draft=draft))       # only reached if gates passed
# Gates are cheap: a code assertion, a regex, a schema (Pydantic), or a small
# LLM-as-judge call. On failure they RETRY, ESCALATE, or HALT.

A gate can be cheap code (a schema/regex/assertion) or a small LLM-as-judge call. On failure it does one of three things: retry the step, escalate to a better model or a human, or halt so a bad output never ships. The guidance from every practitioner source is unambiguous: validation gates are reliability engineering, not optional. "Design quality gates between chain steps that catch errors before they cascade downstream."

This is the single highest-value habit in chaining: validate at every handoff. A gate is what turns "I hope each step worked" into "I verified each step before trusting it."

See the Chain Run (and Watch a Gate Save You)

Step the chain below through a real task — a messy refund email → extract → draft → translate. Then flip the two toggles and watch the core lesson happen in front of you: inject a fault in the draft step, and turn the gates on vs. off.

Step a messy refund email through a 3-step chain (extract → draft → translate). Then flip two toggles: inject a fault in the draft step, and turn the gates on/off. Watch the gate catch the flaw and short-circuit (retry) — versus the flaw amplifying down the chain and shipping a $9,999 promise to the customer when gates are off.

Run all four combinations and feel the difference:

Clean draft: all three steps pass; each is small and verified at the handoff.
Fault + gates ON: the policy gate catches the bogus $9,999 refund and short-circuits (retry) — the error never reaches step 3 or the customer.
Fault + gates OFF: no gate, so step 3 dutifully translates and ships the $9,999 promise. That's error propagation — one bad step poisoned the whole chain.

The widget is the entire lesson in motion: decompose for accuracy, gate for reliability.

The Self-Correction Chain (the one you'll use most)

Anthropic calls out one chaining pattern as the most common, and it's worth memorizing: self-correction — generate → review → refine. The trick is that the model reviewing its own draft in a fresh call catches mistakes it couldn't see while busy generating:

# The most common chain Anthropic recommends: SELF-CORRECTION
# (generate → review → refine). Each step is a separate call you can log & branch.
draft  = call(f"Write a 100-word product description for: {product}")
review = call(f"Critique this against our style guide; list concrete fixes:\n{draft}\n\nGuide:\n{GUIDE}")
final  = call(f"Rewrite the description, applying this feedback:\n\nDraft:\n{draft}\n\nFeedback:\n{review}")
# The model reviewing its OWN draft in a fresh call catches errors it missed
# while generating — a second pair of eyes, for the price of one more call.

Because each step is a separate call, you can log it, evaluate it, or branch on it — exactly why explicit chaining still earns its place in 2026. A useful nuance from Anthropic: with modern adaptive thinking, the model already does a lot of multi-step reasoning inside one call — so reach for explicit chaining specifically when you need to inspect intermediate outputs, enforce a fixed pipeline structure, or log/branch between steps. (Draft→review→refine is also a preview of the Evaluator–Optimizer pattern in L120 — Evaluator-Optimizer, where the review/refine loop becomes the whole point.)

The Honest Tradeoff (and When NOT to Chain)

Chaining isn't free, and a world-class engineer knows its costs:

Latency & total cost go up. Even though each call is smaller and faster, you're making N sequential calls with N network round-trips — slower end-to-end, and the cumulative token cost can exceed one big call. You're trading latency for accuracy/control — a good trade for quality-critical work, a bad one for a latency-sensitive single lookup.
Error propagation is the real risk. As you saw, an early mistake amplifies down the chain. This is why gates aren't optional — and why you should abort on a critical-step failure rather than propagate a bad output.
More moving parts to maintain. Each step is a prompt to version, test, and monitor.

When to chain: the task decomposes cleanly into fixed, ordered subtasks with linear dependencies — data pipelines (extract → transform → format), content workflows (draft → review → polish), anything where each stage adds value the next depends on.

When NOT to chain: the task is a single focused step already (don't add ceremony to a one-shot classification); the subtasks are independent (that's Parallelization, L118 — run them at once instead of in sequence); or the path is unpredictable (that's an orchestrator or agent).

And one sharp mental-model correction: prompt chaining is not Chain-of-Thought. CoT is reasoning within a single prompt; prompt chaining is multiple separate calls. They're often confused because both have "chain" in the name — but one is a prompting technique inside one call, the other is a pipeline of calls.

Then — and Only Then — the Framework

You just built a chain in plain Python, so a framework will never be a black box to you. Frameworks like LangChain (LCEL) simply wire the same chain with nicer syntax — the | "pipe" operator, where each component consumes the previous one's output:

# First principles first (above) — THEN a framework, with eyes open.
# LangChain LCEL wires the same chain with the `|` "pipe": b consumes a's output.
chain = prompt | llm | output_parser        # a RunnableSequence
result = chain.invoke({"email": email})
#  (LLMChain is deprecated; `prompt | llm` IS the modern chain.)
# Multi-step chains compose these pipes — the EXACT control flow you wrote by
# hand, now with retries, streaming, and tracing included. No magic.

prompt | llm | output_parser builds a RunnableSequence — the exact control flow you wrote by hand. (LangChain's old LLMChain is deprecated; prompt | llm is the modern form.) The framework adds retries, streaming, batching, and tracing for free — useful production conveniences — but the idea is unchanged. Build the chain once by hand; then a framework is just sugar over the loop you already understand. (Same lesson as L111: first principles, then frameworks.)

🧪 Try It Yourself

Reason through these, then use the widget to confirm:

Predict before toggling: in the widget, set fault = on and gates = off. What ends up shipped to the customer, and what's the name for what just happened?
You're turning a research paper into a tweet thread: summarize → extract 5 key points → write 5 tweets. Where would you put a gate, and what would it check?
A teammate's chain occasionally produces a great final answer and occasionally total nonsense, with no pattern. They have no gates. What's the most likely cause, and the one change that fixes it?
When would you collapse a 3-step chain back into a single prompt — i.e., when is chaining over-engineering?
Your manager says "just use chain-of-thought, it's the same thing." Correct them in one sentence.

→ (1) The mistranslated $9,999 refund promise ships — that's error propagation (an early mistake amplifying down the chain because nothing checked the handoff). (2) A gate after extract 5 key points — check there are exactly 5, each grounded in the summary (no hallucinated points) — so the tweets aren't built on invented content. (3) Error propagation through unchecked handoffs: an early step occasionally goes wrong and every later step builds on it. The fix: add validation gates at each handoff (retry/halt on failure). (4) When the task is already easy/atomic, latency matters more than the marginal accuracy, or the steps don't actually depend on validated intermediate outputs — then the extra calls are pure overhead. (5) "CoT is reasoning inside one prompt; prompt chaining is multiple separate calls you can inspect, validate, and branch between — different things."

Mental-Model Corrections

"One perfect mega-prompt is better than several simple ones." Almost always backwards. Decomposition makes each step easier and verifiable — +20–50% accuracy on hard tasks. Mega-prompts that juggle many jobs are the #1 cause of flaky output.
"Prompt chaining = chain-of-thought." No. CoT = reasoning within one prompt; chaining = a sequence of separate calls. Different tools.
"If each step is good, the chain is good." Only with gates. Without them, an early error propagates and amplifies — a chain infects everything downstream (unlike parallel work, where errors isolate).
"Gates are optional polish." Gates are reliability engineering — the difference between a demo and production. Validate every handoff; retry, escalate, or halt on failure.
"Chaining is free accuracy." It trades latency and total cost for reliability. Great for quality-critical pipelines; wasteful for a single atomic task.
"I need LangChain to chain prompts." You need a for-loop and a few call()s. LCEL's | pipe is just sugar over the chain you can write by hand.

Key Takeaways

Prompt chaining = decompose a task into a sequence of focused steps, each processing the previous step's output. Pattern 1 of 5, and the foundation for the rest.
Why: small focused steps are more accurate and verifiable — ~20–50% accuracy gains on hard tasks (50%→95%-reliable). Roots: Least-to-Most / Decomposed Prompting (2022).
Build it from scratch: it's just call()s passing outputs along; use structured handoffs (JSON/XML) so each step parses the last reliably.
Gates make it reliable: a programmatic check between steps (schema / policy / LLM-judge) that retries, escalates, or halts — catching an error before it cascades. Validate every handoff.
Honest tradeoff: more sequential calls → higher latency & cost; an early error amplifies down the chain. Chain for fixed, ordered, dependent subtasks — not for atomic tasks or independent ones.
CoT ≠ chaining; the self-correction chain (draft→review→refine) is the one you'll use most; LCEL's | is just the chain you built by hand.
Next: Routing — when the right first move is to classify the input and send it down a specialized path.