Orchestrator-Worker

Introduction

Pattern 4 of 5 (L115), and the first one where the model — not your code — decides the shape of the work. In Anthropic's words: "In the orchestrator-workers workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results."

Parallelization (L118) also fanned out — but you fixed the subtasks in advance (always run security, perf, tests, style). Orchestrator-workers removes that constraint: it's for the tasks where you can't know the subtasks until you see the input. "Add dark mode" might touch 6 files; "fix the footer typo" touches 1. The orchestrator reads the request and invents the plan.

In this lesson:

The one line that separates this from parallelization (dynamic vs fixed subtasks)
The three roles — orchestrator, workers, synthesizer — and how to build each from scratch
The production instance: Anthropic's multi-agent research system (and the numbers)
The honest cost (~15× a chat), the failure modes, when not to use it, and the framework (LangGraph's Send)

An infographic titled 'Orchestrator-Workers'. A central LLM dynamically breaks down a task, delegates the pieces to worker LLM calls, and synthesizes their results. THE FLOW: a request (add dark mode to the app) goes to an ORCHESTRATOR that inspects it and produces a structured plan — crucially the NUMBER and NATURE of subtasks is decided at runtime from the input (fix a footer typo yields 1 subtask; add dark mode yields about 6: edit theme.css, the ThemeProvider, a toggle component, several page components, tests). WORKERS run each subtask as its own LLM call, in parallel, each with its own context window. A SYNTHESIZER merges the worker diffs into one coherent pull request. THE KEY DISTINCTION FROM PARALLELIZATION: whereas it is topographically similar, the difference is flexibility — subtasks are NOT pre-defined, they are determined by the orchestrator based on the specific input, so you cannot hardcode the fan-out; predictable fan-out means parallelization, data-dependent fan-out means orchestrator-workers. THE PRODUCTION INSTANCE: Anthropic's multi-agent research system uses a lead agent that spawns 3 to 5 subagents with separate context windows that search in parallel, and a multi-agent system with a Claude Opus 4 lead and Claude Sonnet 4 subagents outperformed a single agent by 90.2 percent on their internal research eval. THE HONEST FAILURE AND COST: a flawed plan is a gap every worker inherits (the orchestrator misses a file, or two workers edit overlapping code and their diffs conflict at synthesis), the orchestrator can over-spawn workers, and the synthesizer must reconcile everything; the pattern costs 1 orchestrator plus N workers plus 1 synthesizer and multi-agent systems use about 15 times more tokens than a chat, so it is justified only for high-value, genuinely unpredictable tasks. It is not parallelization (fixed subtasks), not routing (which picks one known path), and not a full multi-agent system (these workers are stateless and ephemeral). Takeaway: reach for the orchestrator only when you truly cannot predict the decomposition — the LLM invents the subtasks, runs them as parallel workers, and synthesizes, and you pay about 15 times the cost so it is worth it only when the task value and unpredictability demand it.

The One Line That Separates It From Parallelization

If you remember one sentence from this lesson, make it Anthropic's:

"Whereas it's topographically similar, the key difference from parallelization is its flexibility — subtasks aren't pre-defined, but determined by the orchestrator based on the specific input."

Same shape (a fan-out of workers + an aggregation), but who decides the branches is different:

Parallelization — you decide the subtasks at design time. The fan-out width N is in your code. (You always run the same 4 PR reviewers.)
Orchestrator-workers — the LLM decides the subtasks at runtime, from the input. N is a variable in state, not a constant you wrote.

And Anthropic's exact when-to-use:

"This workflow is well-suited for complex tasks where you can't predict the subtasks needed (in coding, for example, the number of files that need to be changed and the nature of the change in each file likely depend on the task)."

The two concrete examples from the article (heading: "Example where orchestrator-workers is useful"): "Coding products that make complex changes to multiple files each time" and "Search tasks that involve gathering and analyzing information from multiple sources for possible relevant information."

The clean mental model: map-reduce where the LLM produces the map set. In classic map-reduce the partition is fixed code; here the list of things to map over is itself a model output — which is exactly why you can't hardcode the loop.

The Three Roles

The pattern is three jobs wired into a fan-out → fan-in:

🧠 Orchestrator — reads the task and emits a structured plan: a list of subtasks (decided dynamically). It's the planner, and it's the single most important call — everything downstream inherits its decomposition.
⚙️ Workers — one focused LLM call per subtask, run in parallel, each with its own context window (so they don't see — or pollute — each other).
🧩 Synthesizer — a final call that takes the original task + all worker outputs and merges them into one coherent result (reconciling overlaps and conflicts, not just concatenating).

Anthropic's cookbook (orchestrator_workers.ipynb) implements exactly this: the orchestrator returns an <analysis> plus a <tasks> list, then a worker runs per task. (Its minimal demo collects the worker outputs rather than fusing them — but the article's definition makes synthesis the third role, and any real system needs it.)

Build It From Scratch — The Orchestrator

The orchestrator's whole job is to turn a task into a typed list of subtasks. Force that with structured output so it can't hand you free text you have to regex:

import anthropic
from pydantic import BaseModel
client = anthropic.Anthropic()

# The ORCHESTRATOR returns a STRUCTURED plan — a LIST of subtasks whose length and
# content depend on the input. You cannot hardcode the fan-out.
class Subtask(BaseModel):
    file: str
    change: str

class Plan(BaseModel):
    subtasks: list[Subtask]          # <- N is decided by the model at runtime

def orchestrate(task: str, repo_map: str) -> Plan:
    return client.messages.parse(    # structured output → schema-guaranteed list
        model="claude-opus-4-8", max_tokens=2048,
        output_format=Plan,
        messages=[{"role": "user",
                   "content": f"Plan the code changes for: {task}\nCodebase:\n{repo_map}"}],
    ).parsed_output
# "fix a typo" → 1 subtask; "add dark mode" → 6. The ORCHESTRATOR decides — not you.

The key line is subtasks: list[Subtask]. Its length isn't fixed — messages.parse (Anthropic's structured-output helper, now GA) guarantees you get a schema-valid Plan, but how many subtasks is the model's call, made from the actual request and codebase. That single property — a model-decided list length — is the entire difference from parallelization, and it's why the next step can't be a gather over a constant.

The Workers (Parallel) and the Synthesizer

Now fan out over whatever the orchestrator produced, run the workers concurrently (they're independent — perfect for asyncio.gather), and fan back in through a synthesizer:

import asyncio
aclient = anthropic.AsyncAnthropic()

# Each WORKER is a focused call for ONE subtask — its OWN context, run in PARALLEL.
async def work(sub: Subtask) -> str:
    msg = await aclient.messages.create(
        model="claude-sonnet-4-5", max_tokens=2048,     # a cheaper model for workers
        messages=[{"role": "user",
                   "content": f"Make ONLY this change.\nFile: {sub.file}\nChange: {sub.change}"}],
    )
    return msg.content[0].text

async def run_workers(plan: Plan) -> list[str]:
    return await asyncio.gather(*(work(s) for s in plan.subtasks))   # N at once

# The SYNTHESIZER holds the original task + ALL worker diffs and merges them into
# one coherent result (reconciling conflicts — not just concatenating).
def synthesize(task: str, diffs: list[str]) -> str:
    joined = "\n\n".join(f"--- diff {i} ---\n{d}" for i, d in enumerate(diffs))
    return client.messages.create(
        model="claude-opus-4-8", max_tokens=4096,
        messages=[{"role": "user",
                   "content": f"Task: {task}\n\nWorker diffs:\n{joined}\n\nAssemble one coherent PR."}],
    ).content[0].text

# End to end — 1 orchestrator + N workers (parallel) + 1 synthesizer:
plan  = orchestrate(task, repo_map)     # 1 call  → list[Subtask]  (the dynamic plan)
diffs = asyncio.run(run_workers(plan))  # N calls, concurrent      (the fan-out)
pr    = synthesize(task, diffs)         # 1 call  → the assembled PR (the fan-in)

Three things worth internalizing:

Workers get their own context. Each sees only its subtask — so a 6-file change doesn't force one giant prompt, and each worker stays focused (and cheap; use a smaller model here).
Parallel buys latency, not cost. The N workers run at once (wall-clock ≈ the slowest), but you still pay for every token (L118's lesson).
The synthesizer is real work. It must hold all worker outputs and reconcile them — that's where conflicts get resolved and where a weak merge can waste good per-worker output. Anthropic even splits a separate citation pass out of synthesis in production.

See It Decompose (and Watch a Plan Go Wrong)

Pick a coding request below and watch the orchestrator produce a different-sized plan each time — that's the dynamic fan-out, live. Then flip the decomposition to flawed and see how one bad planning decision poisons everything downstream:

Pick a coding request and watch the orchestrator generate a DIFFERENT plan each time — “fix the footer typo” → 1 subtask, “add dark mode” → 6, “add CSV export” → 4. The fan-out is data-dependent (you can't hardcode it — that's why it's not parallelization). Workers run the subtasks in parallel; a synthesizer assembles the PR. Then flip the decomposition to “flawed” and watch a single missed file become a gap that ships a half-broken build — the root failure mode the whole pattern inherits.

Two beats to feel:

Switch requests. "Fix the footer typo" → 1 subtask; "add dark mode" → 6. Same machinery, data-dependent fan-out. You could never have hardcoded that N — which is the whole reason this isn't parallelization.
Flip to a flawed plan. The orchestrator misses Dashboard.tsx, so it never becomes a worker — and dark mode ships half-broken. A flawed plan is a gap every worker inherits; the workers did their jobs perfectly and the result is still wrong. That's the pattern's root failure mode.

The Production Instance — Anthropic's Multi-Agent Research System

Orchestrator-workers isn't a toy — it's the backbone of Anthropic's multi-agent research system (what powers Claude's Research). A lead agent (orchestrator) "analyzes [the query], decides on an overall strategy, and records the plan," then spawns subagents (workers) that explore in parallel.

The numbers, straight from their engineering write-up:

It works: "a multi-agent system with Claude Opus 4 as the lead agent and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval."
Why subagents: they "operat[e] in parallel with their own context windows" — separate context is how you avoid overflowing one giant agent, and parallelism "cut research time by up to 90%" on complex queries (the lead spins up 3–5 subagents, each using 3+ tools in parallel).
The cost: "agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats" — and "token usage by itself explains 80% of the variance" in research success. More tokens is most of the performance.

One nuance for vocabulary: this workflow pattern usually means one system with stateless, ephemeral workers. Anthropic's research system pushes it toward a true multi-agent system (subagents are autonomous, run their own loops). Same orchestrator-worker topology; the difference is how much state and autonomy the workers have. It's a spectrum, not a wall.

The Honest Cost, the Failure Modes, and When NOT to Use It

This is the most expensive and least predictable of the five patterns. Respect its costs:

Bad decomposition (the root failure). The plan is a single point everything inherits. A bad split means gaps (a missed file → broken build, as in the widget) or overlaps (two workers edit the same code → their diffs conflict at synthesis). Anthropic saw early agents "duplicate work, leave gaps, or fail to find necessary information."
Runaway spawning. The orchestrator can over-decompose — Anthropic literally saw agents "spawning 50 subagents for simple queries." Cap the worker count; add budget ceilings.
The synthesizer bottleneck. Merging many (possibly conflicting) outputs is hard, and the synthesizer must hold them all — context pressure plus a single point where good work can be wasted.
~15× the tokens, plus non-determinism: the plan differs run-to-run, so it's harder to test and debug than a fixed workflow. (The MAST study, Why Do Multi-Agent LLM Systems Fail?, catalogs 14 failure modes, with ~44% rooted in poor system/task design.)

When to use it: the decomposition is genuinely unpredictable and the task is high-value — multi-file refactors, deep research, anything where you truly can't enumerate the subtasks in advance. Anthropic's bar: "multi-agent systems require tasks where the value of the task is high enough to pay for the increased performance."

When NOT to: you can predict the subtasks → use Parallelization (cheaper, deterministic). The task is simple or a fixed pipeline → chaining or a single call. Cost/latency matters and the value doesn't justify N+2 calls. Or the subtasks have tight interdependencies / shared context — Anthropic notes the pattern "struggles with domains that require all agents to share the same context."

Then — and Only Then — the Framework

You built the fan-out by hand, so the framework won't be a black box. LangGraph is the canonical implementation, and it needs the Send API precisely because a normal edge can't emit a runtime-variable number of branches:

# THE canonical framework example. Dynamic fan-out needs the Send API — a normal
# edge can't emit a RUNTIME-VARIABLE number of branches.
from langgraph.types import Send                 # (current path; old tutorials use langgraph.constants)
from typing import Annotated
import operator

class State(TypedDict):
    task: str
    subtasks: list[Subtask]
    done: Annotated[list, operator.add]          # a REDUCER merges the parallel workers

def assign_workers(state):                       # one parallel branch PER subtask
    return [Send("worker", {"sub": s}) for s in state["subtasks"]]

builder.add_conditional_edges("orchestrator", assign_workers, ["worker"])
builder.add_edge("worker", "synthesizer")
# orchestrator (structured plan) → N workers (Send fan-out) → synthesizer (reads `done`).
# The `Send` list IS the dynamic decomposition; operator.add is the fan-in.

assign_workers returns a list of Send — one per subtask the orchestrator produced — and that list is the dynamic decomposition. The Annotated[list, operator.add] reducer is the fan-in: parallel workers append to shared state instead of clobbering it. (Heads-up: the current import is from langgraph.types import Send; older tutorials use langgraph.constants — a common breakage.) Build the orchestrate→fan-out→synthesize loop once by hand; the framework just gives it durable state and a graph to draw. (Same lesson as L111 — Building a ReAct Agent From Scratch/L116–L118 — Prompt Chaining → Parallelization.)

🧪 Try It Yourself

Reason through these, then use the widget to confirm:

Predict before clicking: in the widget, how many subtasks does "fix the footer typo" produce vs "add dark mode"? What's the name for the property that makes those two numbers different — and why does it rule out parallelization?
You're generating a weekly report with exactly four fixed sections every time (summary, metrics, risks, next steps). Orchestrator-workers or parallelization — and why?
In the widget, flip "add dark mode" to the flawed decomposition. The workers all succeed, yet the result is broken. Where did the failure actually happen, and what's the one-line name for it?
Your orchestrator-worker system costs ~15× a single call. Name two conditions that must both hold for that to be worth it.
A teammate built an orchestrator that, for a trivial query, spawned 40 workers. What guardrail was missing, and what's the broader lesson about the orchestrator's plan?

→ (1) Footer typo → 1, dark mode → 6. The property is the data-dependent (dynamic) fan-out — the orchestrator decides N at runtime from the input. Parallelization requires a fixed N you write in code, which can't express "1 sometimes, 6 other times." (2) Parallelization — the four sections are known and fixed, so just run them concurrently; a runtime planning call would add cost and non-determinism for zero benefit. (3) In the orchestrator's plan — it left Dashboard.tsx out, so it never became a worker. Bad decomposition (a gap): a flawed plan is inherited by every worker, and perfect workers on an incomplete plan still ship a broken result. (4) The decomposition is genuinely unpredictable (you can't enumerate the subtasks ahead of time) and the task is high-value enough to justify the 15× spend. (5) A cap on the number of workers (plus a cost/budget ceiling). Broader lesson: evaluate the orchestrator's plan as its own artifact — it's the single point of failure the whole fan-out depends on.

Mental-Model Corrections

"Orchestrator-workers is just parallelization." No — who decides the subtasks differs. Parallelization: you, at design time (fixed N). Orchestrator-workers: the LLM, at runtime (dynamic N). That one line is the whole pattern.
"The orchestrator does the work." It plans and delegates. The workers do the work; the synthesizer merges it. Three roles, not one.
"More workers = better." Not free — each is tokens, and the orchestrator can over-spawn (Anthropic saw 50 for a simple query). Cap it.
"If every worker succeeds, the result is right." Only if the plan was right. A gap or overlap in the decomposition is inherited by everyone — perfect workers on a flawed plan still ship a broken build.
"It's basically a multi-agent system." Same topology, but the workflow version uses stateless, ephemeral workers; a full multi-agent system has persistent, autonomous agents. Spectrum, not synonym.
"Reach for it whenever a task is complex." Only when you can't predict the decomposition and the task is high-value — it's ~15× the cost and non-deterministic. If you can enumerate the subtasks, parallelize.

Key Takeaways

Orchestrator-workers = a central LLM dynamically decomposes a task, delegates to parallel worker calls, and synthesizes the results. Pattern 4 of 5.
The one line vs parallelization: subtasks are not pre-defined — the orchestrator decides N (and what each is) at runtime, from the input. Data-dependent fan-out; you can't hardcode it. Map-reduce where the LLM makes the map set.
Three roles: orchestrator (structured plan) → workers (one focused call each, own context, run in parallel) → synthesizer (merge into one coherent result).
Build it: force the plan with structured output (list[Subtask]), asyncio.gather the workers, then a synthesizer call. Total = 1 + N + 1 calls.
Production proof: Anthropic's multi-agent research system — lead + 3–5 subagents, separate context windows, +90.2% vs single-agent — but ~15× the tokens (and tokens explain ~80% of the gain).
Failure modes: a flawed plan (gaps/overlaps) every worker inherits, runaway spawning, the synthesizer bottleneck, and non-determinism. Use it only when the decomposition is unpredictable and the task is high-value; otherwise parallelize or chain.
Framework: LangGraph's Send API does the dynamic fan-out; operator.add reduces the fan-in.
Next: Evaluator-Optimizer (L120) — the last pattern, where one LLM generates and another critiques in a loop until the output clears a bar.