Prompt Chaining & Task Decomposition

Introduction

You've now seen three ways to squeeze more out of a single model call: chain-of-thought (reason before answering), self-consistency (vote across reasonings), and ReAct (reason and act in a loop). Each pushes harder on one call — or one model-driven loop.

But some tasks are simply too big for one prompt. Ask a model to "research this topic, outline an article, write it, fact-check it, and fix the tone" all at once and quality sags: it juggles five jobs, quietly drops half the instructions, and when the result is wrong you can't tell which part failed.

The fix is the oldest move in engineering: decompose. Break the big job into small, focused steps, then chain them — each step's output becomes the next step's input. And here's the shift from last lesson: unlike ReAct, where the model decides each move, you, the engineer, define the pipeline in code. That's the line between a workflow and an agent — and it's the backbone of reliable LLM systems.

In this lesson you'll learn:

Why one monolithic prompt loses to a chain of focused ones
Task decomposition — splitting a job into clean subtasks
Prompt chaining — wiring each step's output into the next
Gates — cheap programmatic checks between steps that catch failures early
Chaining vs ReAct (workflow vs agent) — and when to reach for each

The Problem: One Prompt Doing Too Much

Stuffing a whole task into one prompt feels efficient. It isn't. A single call asked to draft + critique + rewrite + format has to hold every instruction at once, and a few predictable things go wrong:

Instructions get dropped. The more you ask for in one breath, the more likely the model silently ignores part of it.
Quality averages out. Attention is split across sub-tasks; none gets the model's full effort.
It's a black box. When the output is wrong, you can't see which sub-task failed — was the analysis bad, or just the formatting? You can only re-roll the whole thing.
You can't evaluate the pieces. A monolith has one giant input and output; you can't measure or improve a single stage.

Controlled studies back this up: a chain of short, focused prompts is more accurate and more consistent than one mega-prompt trying to do everything — because each shorter instruction is less ambiguous and gets the model's full attention. (We'll measure exactly this kind of difference in the Evaluation container.)

Task Decomposition: Break It Into Focused Steps

Task decomposition is the move: split a complex job into a sequence of simpler subtasks, each of which the model can do well. Instead of one prompt that does everything, you get a handful of prompts that each do one thing.

Take "write a short explainer article." Decomposed:

Outline — produce a tight structure first.
Draft — expand the outline into prose.
Polish — tighten the length and tone.

Each step is now an easy call: "just outline," "just draft from this outline," "just polish." The model isn't juggling — and you can inspect, test, and improve each stage on its own. It's the same "make each call easier" principle behind chain-of-thought, but applied across calls instead of within one. (Researchers call the general idea least-to-most prompting: solve the simpler sub-steps first, then build on them.)

Prompt Chaining: Each Step Feeds the Next

Decomposition gives you the steps; prompt chaining wires them together: the output of each LLM call becomes (part of) the input to the next. The outline flows into the draft; the draft flows into the polish. The result is a pipeline — a fixed path you define in code.

This is exactly the prompt chaining workflow pattern from Anthropic's Building Effective Agents: "decompose a task into a sequence of steps, where each LLM call processes the output of the previous one." It's worth naming what kind of system this is. Anthropic draws a sharp line:

Workflows — "systems where LLMs and tools are orchestrated through predefined code paths" (you write the control flow).
Agents — "systems where LLMs dynamically direct their own processes and tool usage" (the model writes the control flow — like ReAct).

Prompt chaining is a workflow: you decide the steps and their order; the model just executes each one. That predictability is the whole point — it's what makes chains reliable, debuggable, and cheap to evaluate. (Prompt chaining is one of five named workflow patterns; we'll formalize the full set — routing, parallelization, orchestrator-worker, evaluator-optimizer — in Container 3.)

An infographic titled 'Prompt Chaining: Decompose, Then Chain'. A left-to-right pipeline: an Input card (brief: explain vector databases to a beginner, about 150 words) flows into Step 1 Outline (an LLM call), then into a Gate that checks the outline has at least 3 points (pass continues right; fail loops back to fix step 1), then into Step 2 Draft (an LLM call), then Step 3 Polish (an LLM call), ending in an Output card (a finished beginner-friendly explainer). Each step is labelled as one focused LLM call; the gate is labelled plain code, not a model call. A note contrasts this developer-defined workflow path with ReAct's model-driven loop. A bottom banner says each call does one easy job, so the chain is more accurate, debuggable and cheap to evaluate, trading a little latency for reliability.

Gates: Validate Between Steps

Because the steps run in your code, you can drop a gate between any two of them — a programmatic check on an intermediate result before it flows onward. Anthropic calls these "gates"; they're plain code, not extra model calls.

A gate on our chain: after the outline step, verify it actually is a usable outline (enough points, covers the brief) before spending a call to draft from it. If it fails, retry the outline or stop — cheaply, early, before the error propagates into a polished-but-wrong article.

Treat each step's output like a contract: define the shape it must have, and check it before accepting. Gates are where prompt chains earn their reliability — they turn a fragile sequence into one that fails loudly and early instead of silently producing garbage at the end.

Hands-On: Build a Chain From Scratch

Let's build the chain with raw API calls — no framework. Start with the gate, because it's the part most tutorials skip and the easiest to get right: it's not an LLM call at all, just ordinary code that inspects the previous step's output. Because it's deterministic, you can unit-test it (and run it right here):

def gate_outline(outline: str) -> bool:
    """A 'gate' is a plain-code check BETWEEN llm steps — no model call."""
    points = [ln for ln in outline.splitlines() if ln.strip().startswith(("-", "*"))]
    return len(points) >= 3              # need >= 3 outline points before drafting


# self-contained — run it
sample = "- What it is\n- Why it matters\n- How it works\n- Example"
print(gate_outline(sample))              # -> True
print(gate_outline("- only one point"))  # -> False

Now the chain itself — three focused calls, with the gate sitting between step 1 and step 2. Each function does exactly one job, and each output flows into the next:

from anthropic import Anthropic

client = Anthropic()  # reads ANTHROPIC_API_KEY

def llm(prompt: str, system: str = "You are a precise writing assistant.") -> str:
    r = client.messages.create(
        model="claude-sonnet-4-6", max_tokens=700,
        system=system, messages=[{"role": "user", "content": prompt}],
    )
    return r.content[0].text

# --- three focused steps: each call does exactly ONE job ---
def make_outline(brief: str) -> str:
    return llm(f"Write a 4-point outline for: {brief}\n"
               f"Return ONE '- ' bullet per line, nothing else.")

def write_draft(brief: str, outline: str) -> str:
    return llm(f"Brief: {brief}\nOutline:\n{outline}\n\n"
               f"Write the full piece, following the outline.")

def polish(draft: str) -> str:
    return llm(f"Tighten this to ~150 words, clear and friendly:\n\n{draft}")

# --- the chain: each output flows into the next, with a GATE in the middle ---
brief = "Explain vector databases to a beginner."
outline = make_outline(brief)                       # step 1
if not gate_outline(outline):                       # gate (plain code, from above)
    raise ValueError("Outline failed the gate — fix step 1 before drafting.")
draft = write_draft(brief, outline)                 # step 2
final = polish(draft)                               # step 3
print(final)

Three small, easy calls instead of one impossible one. If the article comes out wrong, you can look at the outline, the draft, and the polish separately and fix the exact stage that failed — and the gate guarantees you never draft from a broken outline.

The pattern is provider-agnostic — the orchestration lives in your code, not the model. In OpenAI only the llm helper changes:

# The same helper with OpenAI — only the call changes:
from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY

def llm(prompt: str, system: str = "You are a precise writing assistant.") -> str:
    r = client.chat.completions.create(
        model="gpt-5.5",
        messages=[{"role": "system", "content": system},
                  {"role": "user", "content": prompt}],
    )
    return r.choices[0].message.content

Chaining vs ReAct — Workflow vs Agent

Both chaining and ReAct break a big task into smaller moves. The difference is who decides the moves:

	Prompt Chaining (workflow)	ReAct (agent)
Who decides the steps	You, in code — a fixed path	The model, at runtime — dynamic
Best when	Task splits into known, fixed subtasks	Path is unknown; needs to adapt/explore
Reliability	High — predictable, gated, testable	Lower — flexible but can wander
Debug / eval	Easy — inspect each step	Harder — variable trajectory
Cost / latency	Fixed number of calls	Variable (loops until done)

Rule of thumb: if you can draw the flowchart, chain it; if you can't, you may need an agent. Most production LLM features are chains (sometimes with a little branching), not open-ended agents — chains are cheaper, safer, and easier to trust. Reach for an agent only when the task genuinely requires the model to choose its own path.

Pitfalls & Common Mistakes

Over-decomposing. Ten micro-steps means ~10× the latency and cost for little gain. Split only where a step is genuinely a distinct job.
No gates → silent error propagation. A bad early output quietly poisons every step after it. Validate between stages so failures surface immediately.
Dropping context between steps. Each call only sees what you pass it — forward the brief and constraints, or step 3 forgets what step 1 knew.
Chaining when one prompt would do. If a single focused prompt nails it, a chain is just extra latency. Decompose because the task is too big, not by reflex.
Treating the chain as frozen. Evaluate each step; one weak link (say, the outline stage) drags the whole pipeline down — fix that stage, not the monolith.

🧪 Try It Yourself: Step Through the Chain

Prompt chaining is a pipeline — best understood by walking it. Step through the chain below one stage at a time: watch each step's output become the next step's input, and see how the gate sits between steps as plain code (not a model call).

Then build your own. Take a task you'd normally one-shot — "turn these messy meeting notes into a polished follow-up email" — and split it into a 3-step chain: (1) extract the action items as a list, (2) draft the email from that list, (3) rewrite it in a warm, concise tone. Run the three prompts in sequence (paste each output into the next), then compare against a single mega-prompt that asks for all three at once. When something's off, which version lets you pinpoint and fix the exact step that failed?

Chain Runner — run a 3-step writing pipeline (Outline → gate → Draft → Polish). Set the outline quality (thorough or thin) and the mode (a gated chain vs one mega-prompt). A thin outline trips the gate (a plain-code check, not an LLM call) which stops the chain before wasting the draft and polish calls — caught early and cheaply — while a thorough outline flows all the way to a debuggable, shipped result. Switch to one mega-prompt and it silently drops an instruction and becomes a black box you cannot debug and can only re-roll. Shows why decomposing into a gated chain beats a monolithic prompt: failures surface early, and every stage is inspectable.

Key Takeaways

Some tasks are too big for one call — decompose them into focused subtasks, then chain them so each step's output feeds the next.
A chain of short, focused prompts beats one monolithic prompt: more accurate, more consistent, and debuggable step-by-step.
Prompt chaining is a workflow (you define the path in code) — distinct from an agent like ReAct (the model defines the path). If you can draw the flowchart, chain it.
Gates — cheap programmatic checks between steps — are where chains get their reliability: they catch failures early, before they propagate.
It trades a little latency (more calls) for accuracy and control. Don't over-decompose, and pass enough context between steps.

Next: Meta-Prompting & Reusable Templates — instead of hand-writing every prompt in a chain, we'll use prompts that generate and standardize other prompts.