Supervisor / Hierarchical Architectures

Introduction

Last lesson asked whether you need more than one agent (L143 — Why Multiple Agents?). Suppose you've decided you do — the task is genuinely parallel, spans real specialties, and is worth the cost. Now the engineering question: how do you wire those agents together?

Who talks to whom? Who's in charge? Who decides what runs next, and who assembles the final answer? The shape of those connections is your system's topology — its org chart. Pick the right one and the system is reliable, traceable, and debuggable. Pick the wrong one and you've rebuilt exactly the chaos L143 (Why Multiple Agents?) warned about: agents making conflicting decisions no one can reconcile.

The dominant, start-here topology is the supervisor: one agent that delegates to specialists and stitches their work back together. Stack supervisors and you get a hierarchy — teams of teams. These centralized shapes run the large majority of multi-agent systems shipping in 2026.

In this lesson:

The three topologies — network, supervisor, hierarchical — and which to reach for (hands-on)
The supervisor pattern — one orchestrator, workers that report back, a single thread of control
How it delegates in code — agents-as-tools, handoff tools, and the frameworks
Anthropic's research system — a supervisor in production (+90%)
Going hierarchical, and when the supervisor becomes the bottleneck

Scope: this lesson is the centralized topologies. The peer-to-peer handoff / network model and shared state are next (L145 — Agent Handoffs & Shared State); the open A2A protocol follows (L146 — The A2A (Agent-to-Agent) Protocol); the deep coordination & cost math is L147 (Coordination & Cost); and designing a full system end-to-end is L148 (Designing a Multi-Agent System).

An infographic titled 'Supervisor and Hierarchical Architectures' showing how to wire a team of AI agents together. The center contrasts three topologies drawn as graphs. The NETWORK topology is a mesh: five agent nodes each connected to every other, a tangle of lines, labelled 'every agent talks to every agent' and marked as the peer-to-peer or handoff model. The SUPERVISOR topology is a star: one supervisor node at the top connected to four worker nodes below it, with arrows going down for 'delegate' and up for 'report back', labelled 'one orchestrator, single thread of control'. The HIERARCHICAL topology is a tree: a top orchestrator connected to two team leads (sub-supervisors), each of which connects to two workers, labelled 'teams of teams'. A panel explains the supervisor pattern: the supervisor reads the goal, decomposes it, delegates each piece to a specialist worker that runs in its own isolated context window, collects the results, and synthesizes the final answer, so every decision flows through a single agent. A production example is shown: Anthropic's research system, where a lead agent (Claude Opus) spawns three to five subagents (Claude Sonnet) that search in parallel and report back, beating a single agent by ninety percent on broad research. Cards give the rules: start with a supervisor because it is the simplest topology that still wins and the easiest to debug; go hierarchical only when distinct sub-domains each need their own specialist team, since every layer adds a planning round-trip in latency and cost; and watch the supervisor bottleneck, where the single orchestrator becomes a context and reasoning bottleneck and a single point of failure, mitigated with scoped budgets, external memory, and splitting into a hierarchy as the team grows past about seven workers. The takeaway: the supervisor, one agent that delegates and synthesizes, is the default multi-agent architecture; add hierarchy only when the work genuinely splits into specialized teams.

The Org Chart — Three Topologies

Strip away the details and every multi-agent system is one of three shapes — a spectrum from decentralized to centralized:

Network (a mesh): every agent can talk to every other. Maximum flexibility, maximum chaos — N agents means up to N(N−1)/2 communication paths, and no one owns the outcome. This is the peer-to-peer / handoff model (next lesson, L145 — Agent Handoffs & Shared State).
Supervisor (a star): one orchestrator sits in the middle; each worker connects only to it. The supervisor decides who does what and merges the results. N workers = just N paths.
Hierarchical (a tree): a supervisor of sub-supervisors. Each team lead runs its own workers; the top orchestrator coordinates the leads.

The trade-off is flexibility vs. control. The network maximizes flexibility (and disorder); the hierarchy maximizes control (and overhead); the supervisor is the pragmatic middle that most systems should start from. Switch between them and watch the graph — and the cost of coordination — change:

The three multi-agent topologies, side by side. Toggle NETWORK (a mesh — every agent talks to every agent, up to N(N-1)/2 paths, no one owns the outcome; this is the peer/handoff model of L145), SUPERVISOR (a star — one orchestrator delegates to workers that report back, N paths, a single thread of control, the default to start from), and HIERARCHICAL (a tree — a supervisor of sub-supervisors, teams of teams, for when sub-domains diverge). The graph shape morphs and the metrics — communication paths, who decides, single point of failure, debuggability, best-for — and a 'run a task' trace update with each. Start with a supervisor; go hierarchical only when distinct sub-domains each need their own specialist team.

Keep this map in your head for the rest of the section: we now zoom into the two centralized shapes — the supervisor and its stacked form, the hierarchy. (The mesh/handoff model gets its own lesson, L145 — Agent Handoffs & Shared State.)

The Supervisor Pattern

Zoom into the star. A supervisor — also called an orchestrator, lead, manager, or router — is an agent whose job is coordination, not execution. It reads the goal, decomposes it, delegates each piece to a specialist worker, collects the results, and synthesizes the final answer.

The workers are themselves agents — each with its own prompt, its own tools, and crucially its own isolated context window (so one worker's noise never crowds another's, a direct payoff of the memory lessons, L132 — Why Agents Need Memory). But the workers only ever talk to the supervisor, never to each other.

The defining property is a single thread of control: every decision flows through one agent. That's exactly what addresses the failure L143 (Why Multiple Agents?) warned about — dispersed decisions that conflict and can't be reconciled (Cognition's "Don't Build Multi-Agents" argument). Centralize the decisions in a supervisor and they stay consistent.

You've actually seen the skeleton before: the orchestrator–worker workflow (L119). The supervisor is that same idea promoted from a fixed workflow to a full architecture — the workers aren't predetermined code steps anymore; they're autonomous agents the supervisor chooses among, and briefs, at runtime.

How a Supervisor Delegates

Under the hood, a supervisor delegates in one of two ways — and the distinction matters when you debug one:

Agents-as-tools (the manager): each worker is wrapped as a tool (Section 3, L121 — Tools). The supervisor calls a worker exactly like any other tool, the worker runs, and its result comes back to the supervisor — which stays in control and does the final synthesis. This is the cleanest mental model and the easiest to debug: every routing decision lives in one place.
Handoff / routing: instead of calling a worker and getting a value back, the supervisor outputs which agent should run next, and a graph transfers control there. LangGraph's create_supervisor builds this with handoff tools (e.g. assign_to_research_expert).

Here's the agents-as-tools supervisor from scratch. Notice it's just the tool-use loop you already know (L121, L141 — Connecting an Agent to MCP Servers) — except the "tools" happen to be other agents:

# A SUPERVISOR is just an agent whose "tools" are other agents. It delegates,
# collects results, and synthesizes — the tool-use loop of L121/L141, one level up.
from anthropic import Anthropic
client = Anthropic()

def worker(system, task):              # a worker = its own focused agent (own prompt, own context)
    m = client.messages.create(model="claude-opus-4-8", max_tokens=1024,
            system=system, messages=[{"role": "user", "content": task}])
    return m.content[0].text

WORKERS = {                            # each specialist, with a distinct role
    "research": lambda t: worker("You are a meticulous researcher. Cite sources.", t),
    "write":    lambda t: worker("You are a crisp technical writer.", t),
}
# Expose each worker to the supervisor as a TOOL (Section 3):
WORKER_TOOLS = [{"name": k, "description": d,
    "input_schema": {"type": "object", "properties": {"task": {"type": "string"}},
                     "required": ["task"]}}
    for k, d in [("research", "Delegate a research subtask. Returns facts + sources."),
                 ("write",    "Delegate a writing subtask. Returns drafted prose.")]]

# The SUPERVISOR LOOP — it decides who runs, in what order, then writes the answer:
messages = [{"role": "user", "content": "Write a short brief on MCP security."}]
while True:
    m = client.messages.create(model="claude-opus-4-8", max_tokens=1024,
            system="You are a supervisor. Delegate to your worker tools, then "
                   "synthesize their results into the final answer yourself.",
            messages=messages, tools=WORKER_TOOLS)
    if m.stop_reason != "tool_use":
        print(m.content[0].text); break            # the supervisor synthesized — done
    for b in (c for c in m.content if c.type == "tool_use"):
        result = WORKERS[b.name](b.input["task"])   # the worker agent runs in ITS own context
        messages += [{"role": "assistant", "content": m.content},
                     {"role": "user", "content": [{"type": "tool_result",
                        "tool_use_id": b.id, "content": result}]}]
# Frameworks wrap exactly this: LangGraph create_supervisor([research, write]),
# CrewAI Process.hierarchical (manager_llm=...), the OpenAI Agents SDK (agents as tools).

That's the whole pattern: a supervisor LLM, a set of worker agents exposed as tools, a loop that delegates → runs the worker → feeds the result back → synthesizes. Production frameworks wrap precisely this — LangGraph (create_supervisor), CrewAI (Process.hierarchical with a manager_llm), the OpenAI Agents SDK (agents-as-tools), and the Claude Agent SDK — so you rarely hand-roll the loop, but now you know what they're doing when a worker doesn't get called or a result goes missing.

A Supervisor in Production — Anthropic's Research System

The clearest real-world supervisor is Anthropic's multi-agent research system. A lead agent (Claude Opus) receives a broad query, plans a strategy, and spawns 3–5 subagents (Claude Sonnet) that search in parallel, each in its own context window. The subagents act as intelligent filters — they do the heavy searching and return only what matters — and the lead synthesizes the final report. On breadth-first research, that system beat a single agent by 90.2%.

The hard-won lesson wasn't about the topology; it was about delegation quality. Vague briefs like "research the semiconductor shortage" made subagents duplicate each other's work and leave gaps. Delegation is a prompt — each worker needs a crisp, bounded brief:

# DELEGATION IS A PROMPT. Anthropic's hard-won lesson: vague briefs like
# "research the semiconductor shortage" made subagents duplicate work and leave gaps.
# A good supervisor spends its tokens writing each worker a crisp brief:
subtask = {
    "objective":     "Find the 3 most-cited MCP security incidents of 2025, with sources.",
    "output_format": "Bullets: incident — date — one-line impact — URL.",
    "tools":         ["web_search"],          # which tools / sources to use
    "boundaries":    "2025 only. Catalog incidents; do NOT analyze fixes.",
}
# Objective + format + tools + boundaries = no overlap, no gaps. The supervisor's
# real skill is decomposition and briefing, not micro-managing each step.

Microsoft's Magentic-One shows the same shape with more bookkeeping: a lead Orchestrator maintains a Task Ledger (the facts and the plan) and a Progress Ledger (who's doing what, and are we done yet?), delegating to a web-browsing agent, a coder, and a file agent — and re-planning when a step fails (closing the loop from L131 — Replanning & Recovering From Failure). A supervisor that decomposes well, tracks progress, and re-plans is the workhorse of agentic systems in 2026.

Going Hierarchical — Teams of Teams

One supervisor with a handful of workers covers most systems. You add a layer — a hierarchy — when the work splits into distinct sub-domains that each need their own specialist team. A top orchestrator delegates to team leads (sub-supervisors), and each lead runs its own workers, summarizing up to the orchestrator.

Reach for a hierarchy when:

subtasks genuinely diverge in tools, models, or expertise — e.g. a research team (web search, retrieval) and an engineering team (code execution, tests) want different toolsets and even different models; or
a single supervisor's worker count creeps past ~7, where its context and routing start to degrade — splitting into teams keeps each supervisor's job small enough to do well.

Skip it when the "teams" would all use the same tools and the same model — then the layers buy you nothing but latency. Every level adds a planning round-trip (≈2 seconds and its own tokens), so a three-deep hierarchy can burn several seconds and a real cost premium before a single worker even starts. Add depth only when the specialization is real.

When the Supervisor Becomes the Bottleneck

The supervisor's strength — everything flows through one agent — is also its weakness. The honest failure modes:

Single point of failure. If the supervisor decomposes badly, the whole system fails. Its reasoning is the ceiling on the system's quality.
Context bottleneck. The supervisor must hold the full task, every worker's result, and enough context to synthesize — and that history grows with each round-trip. Past the model's effective window, routing quality drops (Lost-in-the-Middle, L132 — Why Agents Need Memory).
Over-delegation & latency. Every delegation is a round-trip; a chatty supervisor that keeps re-asking workers piles on both cost and seconds.
Information withholding. Because workers don't talk to each other, a fact one worker discovers may never reach another — the supervisor has to relay it, and may not.

Mitigations: give workers scoped budgets (token / iteration caps) so a wanderer can't run away; offload the growing history to external memory (L133); forward a worker's answer straight to the output when it needs no rewriting (saves tokens and avoids misquoting it); and split into a hierarchy before the hub overloads (~7 workers). The bottleneck is real — but it's the price of the consistency that a single thread of control buys you. Choosing the topology is choosing which problem you'd rather have.

Choosing Your Topology

Put it together as a ladder you climb only as far as the task forces you — the same discipline as workflows-vs-agents (L115) and single-vs-multi (L143):

One agent (L143) — the default. Reach for more only when the task is genuinely parallel and specialized.
Supervisor — the start-here multi-agent topology: one orchestrator, specialist workers, a single thread of control. Most multi-agent systems should live here.
Hierarchical — when distinct sub-domains each need their own team; mind the per-layer latency and cost.
Network / handoffs — when peers should pass control directly (triage, escalation) with no central hub — the next lesson (L145 — Agent Handoffs & Shared State).

Each rung up buys capability and costs coordination. Start at the lowest rung that solves the problem, and climb only when the pain is real. A clean supervisor beats a clever hierarchy you didn't need.

🧪 Try It Yourself

Reason these through, then check with the topology explorer:

You're building a system where a front-desk agent classifies a support ticket and sends it to billing, tech, or returns — and that specialist then owns the whole conversation. Which topology is that, and is it this lesson or the next?
A supervisor coordinates 4 workers. How many communication paths are there? Re-wire the same 4 as a network — now how many?
Your supervisor keeps dropping facts one worker found before another needs them. Name the failure mode and one fix.
A teammate proposes a 3-level hierarchy where all 6 leaf agents use the same web-search tool and the same model. What's wrong, and what should they build instead?
Why does a supervisor (single thread of control) directly counter Cognition's "dispersed decisions" critique of multi-agent systems (L143)?

→ (1) A network / handoff topology (the specialist takes over, no central hub, no reporting back) — that's L145 (Agent Handoffs & Shared State), not this lesson; here the front-desk agent would stay in control and call specialists as tools. (2) Supervisor = 4 paths (one per worker). Network = 4×3/2 = 6 paths — and it climbs as N(N−1)/2. (3) Information withholding — workers don't talk to each other, so the supervisor must relay; fix by putting shared findings in a shared scratchpad / external memory (L133) or by having the supervisor explicitly pass each worker the context it needs. (4) The "teams" don't diverge in tools or model, so the layers buy nothing but latency and cost — collapse it to one flat supervisor over the workers. (5) Because every decision flows through one agent, decisions stay consistent — there are no parallel agents making conflicting choices that can't be reconciled; the supervisor is the single decision-maker.

Mental-Model Corrections

"Multi-agent means agents chatting freely with each other." That's the network (mesh) — flexible but chaotic. Most production systems are a supervisor: workers talk only to the orchestrator, never to each other.
"The supervisor is a special kind of agent." It's an ordinary agent running the tool-use loop (L121) — its "tools" are just other agents. Nothing new under the hood.
"This is the same as the orchestrator–worker workflow (L119)." Same shape, different level: L119 (Orchestrator-Worker) was a fixed workflow of code steps; a supervisor's workers are autonomous agents it picks and briefs at runtime.
"More layers = more powerful." Each layer adds a planning round-trip (latency + cost). Add a level only when sub-domains genuinely need different tools/expertise — otherwise it's pure overhead.
"The supervisor just routes." Its hardest job is decomposition and briefing — vague delegation makes workers duplicate work and leave gaps. Delegation is a prompt.
"Centralizing through one agent removes all the failure modes." It trades dispersed failures for a single point of failure and a context bottleneck. You pick which problem you'd rather manage.

Key Takeaways

Topology = your agents' org chart. Three shapes on a decentralized→centralized spectrum: network (mesh — every agent ↔ every agent), supervisor (star — one orchestrator), hierarchical (tree — supervisor of sub-supervisors).
The supervisor is the default. One agent decomposes → delegates to specialist workers → collects → synthesizes, keeping a single thread of control — which tames the dispersed-decisions failure of multi-agent systems (L143).
Mechanically it's the tool-use loop (L121). Agents-as-tools (the manager calls workers, results report back — easiest to debug) or handoff/routing (create_supervisor). Frameworks: LangGraph, CrewAI, OpenAI Agents SDK, Claude Agent SDK.
Delegation is a prompt. Give every worker an objective, output format, tools, and boundaries — vague briefs cause duplicated work and gaps (Anthropic's lead + 3–5 subagents, +90.2%; Magentic-One's task/progress ledgers).
Go hierarchical only when sub-domains diverge in tools/expertise, or a supervisor passes ~7 workers. Each layer costs a planning round-trip — don't add depth you don't need.
Mind the supervisor bottleneck: single point of failure, context bottleneck, over-delegation latency, information withholding. Mitigate with scoped budgets, external memory, message-forwarding, and a hierarchical split.
Climb the ladder only as far as forced: one agent → supervisor → hierarchy → network. Start simple; add structure when the pain is real.
Next — L145: Agent Handoffs & Shared State — the decentralized side: peers passing control directly, and the shared memory that lets them coordinate.