Designing a Multi-Agent System

Introduction

This is the finale of the multi-agent section, and it's the one that ties the rest together. You now have all the pieces: when to use multiple agents (L143), what shape to wire them in (L144), how they coordinate (L145), how they talk across organizations (L146), and what it costs (L147). The question this lesson answers is the engineer's question: how do you put them together into a system you'd actually ship — and keep running?

And there's a single, clarifying finding that should frame everything you do here. When UC Berkeley built MAST — the first taxonomy of why multi-agent systems fail, from 1,600+ real failure traces — the headline was blunt:

Most multi-agent failures are design failures, not model failures. Better base models alone won't fix them. The reliability of a multi-agent system is decided less by which LLM you pick and more by how you designed the system around it — the roles, the boundaries, the verification, the stop conditions.

That's the thesis of the finale: a great multi-agent system is mostly great design. This lesson is the design discipline — a repeatable process, the failure modes to design against, and what it takes to ship.

In this lesson:

The design process — a decision sequence that turns L143–L147 (Why Multiple Agents? → Coordination & Cost) into a system (hands-on)
Why they fail — the MAST taxonomy, and designing against it
Reliability — verifier agents, termination conditions, and guardrails
Engineering lessons from production, plus observability, evaluation, and staged rollout

Scope: this is the capstone of Multi-Agent Systems — it synthesizes the section. From here the course moves to the framework landscape (LangGraph, CrewAI, OpenAI Agents SDK…) that turns these designs into code.

An infographic titled 'Designing a Multi-Agent System' that synthesizes how to put a whole multi-agent system together. The centerpiece is a blueprint of a well-designed system: a lead orchestrator on a capable model coordinates several scoped worker agents on cheaper models, a separate critic or verifier agent checks the workers' outputs, and the whole thing is wrapped in budgets, tracing, and a human approval gate. Each part is annotated with the lesson it draws on: choosing multiple agents at all (when the task is high-value and parallel), the supervisor topology, scoped context for coordination, the A2A protocol only across organizational boundaries, model tiering and budget caps for cost, and a verifier plus explicit termination for reliability. A panel presents the central thesis using the MAST failure taxonomy from Berkeley: most multi-agent failures are design failures, not model failures, with fourteen failure modes in three categories — specification and design issues at about 42 percent, inter-agent misalignment at about 37 percent, and verification failures at about 21 percent — and the worst offenders being task misinterpretation, premature termination, step repetition, and missing verification, none of which a better base model alone will fix. Cards summarize the practice: the design process as a decision sequence from should-you-go-multi through topology, coordination, protocol, cost, reliability, and deploy; designing for reliability with critic agents, explicit termination conditions, and guardrails; and shipping it with staged autonomy from draft-only to execute-with-approval to automatic under thresholds, plus canary rollout, kill switches, tracing, and continuous evals. A note credits engineering lessons from Anthropic: think like your agents, teach the orchestrator to delegate, and scale effort explicitly. The takeaway: a great multi-agent system is mostly great design — decompose well, choose the simplest topology that works, carry context deliberately, control cost, verify outputs, bound every loop, observe everything, and roll out in stages.

The Design Process — A Decision Sequence

Designing a multi-agent system isn't a leap of inspiration; it's a sequence of decisions, each one a lesson from this section. Walk them in order and the architecture designs itself:

Single or multi? Default to one agent. Go multi only if the task is high-value, parallel, and independent (L143). Most of the time you stop here.
Decompose & define roles. Break the task into subtasks with clear responsibilities — a researcher, a writer, a critic. Crisp roles are themselves a guardrail against drift.
Choose the topology. A supervisor by default; a hierarchy only when sub-domains genuinely diverge; network/handoffs for peer hand-offs (L144).
Choose coordination. How does context move — handoffs (structured summaries) and/or shared state (a blackboard)? Carry context deliberately (L145).
Cross any boundaries? If an agent lives in another org/system, use A2A (L146). Inside one codebase, don't.
Cost & controls. Tier models, cache prefixes, scope context, and set budget caps per agent (L147).
Reliability & deploy. Add verification and termination, then trace, evaluate, and roll out in stages (the rest of this lesson).

Watch the sequence build a real system — a market-research assistant — one decision at a time:

Designing a real multi-agent system — a market-research assistant — as a sequence of decisions. Step through it: the task (a broad, parallel, high-value competitive analysis) → Decision 1 single-or-multi (multi, L143) → Decision 2 topology (supervisor + workers, L144) → Decision 3 coordination (structured briefs + a shared scratchpad, not raw web, L145) → Decision 4 boundaries (all in-house, so no A2A; L146) → Decision 5 cost (tier Opus lead + Haiku workers, cache, scope, cap budgets, L147) → Decision 6 reliability (a critic verifies each worker, explicit ≤3-retry termination, effort-scaling rules) → Decision 7 ship (trace every call, eval against gold reports, roll out draft → approval → auto). The result: a supervisor with tiered scoped workers, a verifier, capped budgets, traced and gated — and every decision avoided a known MAST failure mode.

Why Multi-Agent Systems Fail — the MAST Taxonomy

To design against failure, you have to know how these systems actually break. MAST (UC Berkeley) analyzed 1,600+ failure traces across 7 frameworks and found 14 failure modes in 3 categories — and, tellingly, most have nothing to do with the model:

Specification & design issues (~42%) — the biggest bucket. Disobeying the task or role spec, step repetition, losing conversation history, and — the classic — no termination condition (agents that stop too early or never stop). These are architecture bugs.
Inter-agent misalignment (~37%) — failures in how agents talk to each other: failing to ask for clarification, task derailment, withholding context, ignoring each other's outputs. These are coordination bugs (L145).
Verification failures (~21%) — no one checks the work: accepting an unsupported claim, incorrect or skipped verification, declaring success on a wrong answer.

The worst offenders across all three: task misinterpretation, premature termination, step repetition, and missing verification. Notice what they share — they're organizational failures, the kind you'd see in a badly-run human team. That's why MAST's conclusion is so important: "improvements in base model capabilities will be insufficient." You don't fix these with a smarter model; you fix them with better design — which is exactly what the next sections do, category by category.

Designing for Reliability

Map the mitigations straight onto the MAST categories — three moves cover most of the damage:

Add verification (the biggest single lever). Put a critic / judge agent on every critical output — a separate agent that checks the work against the task and the evidence, with explicit pass thresholds and retry limits. Teams report this is the single change that most improves reliability (it directly attacks the ~21% verification bucket). Checks-and-balances generalize: critic agents challenge claims, validator agents enforce constraints, red-team agents probe for failure.
Make termination explicit. The #1 design bug is an undefined stop condition. Every loop needs a hard cap (max steps / max tokens, L147 — Coordination & Cost) and a clear done signal — so agents neither quit early nor spin forever. Pair with a no-progress detector (L131).
Wrap it in guardrails. Constraints, allowlists, validation layers, and scoped budgets keep one confused agent from escalating into a system-wide failure. Task decomposition itself is a guardrail — small, well-specified subtasks give the model less room to wander.

Here's the highest-leverage pattern — a verifier with bounded retries — in code:

# RELIABILITY = checks + bounded loops. The single biggest lever (MAST): a CRITIC that
# verifies critical outputs, with an explicit retry limit and a real termination condition.
from anthropic import Anthropic
client = Anthropic()

def critic(task, output) -> dict:
    """A SEPARATE agent judges the output against the task. Returns {ok, why}."""
    m = client.messages.create(model="claude-opus-4-8", max_tokens=512,
        system='You are a strict reviewer. Reply JSON {"ok": bool, "why": str}. '
               'Fail anything unsupported by evidence or off-spec.',
        messages=[{"role": "user", "content": f"TASK:\n{task}\n\nOUTPUT:\n{output}"}])
    return parse_json(m.content[0].text)

def run_with_verification(task, max_tries=3):
    for attempt in range(max_tries):           # ← explicit termination: never loop forever
        output  = worker_agent(task)
        verdict = critic(task, output)
        if verdict["ok"]:
            return output                       # ← verified — done
        task += f"\n\nReviewer rejected: {verdict['why']}. Fix it and retry."
    raise NeedsHuman("failed verification after retries")   # ← escalate, don't spin
# One pattern attacks three MAST categories at once: VERIFICATION (a judge), premature /
# never-ending TERMINATION (the retry cap), and TASK-SPEC drift (the critic re-checks it).

One small harness, three MAST categories defended at once: a verifier (verification), a retry cap that escalates (termination), and a critic that re-checks the spec (task drift). This is what designing for reliability looks like — not a better model, but a better structure around it.

Engineering Lessons from the Field

When Anthropic took a multi-agent system from prototype to production, the hard-won lessons were almost all about design and prompting, not models. The ones that transfer to any multi-agent system:

Think like your agents. Build a simulation, give it the exact prompts and tools, and watch it work step by step. Most failures are obvious once you see the agent's-eye view — Anthropic spent weeks watching agents fail and rewriting prompts to fix specific modes.
Teach the orchestrator to delegate. A good lead doesn't do everything; it writes each worker a clear brief — objective, boundaries, output format (L145). Delegation is a prompt, and vague delegation is where multi-agent systems rot.
Scale effort explicitly. Agents are terrible at judging how much effort a task needs — early systems spawned 50 subagents for a one-line question. The fix is an explicit rule in the prompt: simple fact-find → 1 agent, 3–10 tool calls; comparison → 2–4 subagents, 10–15 calls each.
Encode expert heuristics. Study how a skilled human does the task (how they decompose, judge sources, choose depth vs breadth) and write those strategies into the prompts.

The thread through all four: the leverage is in the design and the prompts, not the model. A clearly-briefed, bounded, well-observed team of cheaper agents beats a vague swarm of expensive ones.

Observability & Evaluation — You Can't Fix What You Can't See

Multi-agent systems are non-deterministic — the same prompts can produce different runs — which makes debugging by reproduction nearly impossible. The only way to operate them is to see inside them:

Trace everything. Capture the full execution — every agent, prompt, tool call, hand-off, and token count — as a structured trace. Anthropic's single biggest debugging upgrade was full production tracing: it let them diagnose why an agent failed and fix it systematically (and it's how you spot the step-repetition and derailment from MAST).
Evaluate with humans and judges. Build versioned, ground-truth eval sets and run them on every change. Use LLM-as-judge for scale — but human evaluation is irreplaceable for the nuanced failures automation misses (Anthropic caught source-selection bias only by hand). Track cost and latency as first-class eval signals (L147), not just accuracy.
Make evals continuous. Treat evals as something that runs in production and can gate a release on quality drift — not a one-time offline check. Monitor agent decision patterns, not just latency.

The rule: observability is not optional for agents. A multi-agent system you can't trace is a multi-agent system you can't debug, can't evaluate, and shouldn't ship.

Shipping It — Staged Autonomy

The most common production failure isn't in the agents — it's in the rollout. Teams spend weeks on evaluation and then improvise the deploy; the green eval becomes a production incident that a 1% canary would have caught as a blip. Ship a multi-agent system the way you'd ship any high-blast-radius change — gradually, with the human in the loop until trust is earned:

Stage the autonomy. Don't launch full-auto. Roll out draft-only (the agent proposes, a human acts) → execute-with-approval (a human approves each action) → automatic under thresholds (auto for low-risk/high-confidence, escalate the rest). This is human-on-the-loop: autonomous for the routine, human for the edge cases.
Gate the dangerous actions. Even at full auto, high-stakes or low-confidence actions require approval. Writes, sends, and deletes get a gate (echoing the consent model of L142 — The MCP Ecosystem & Security Considerations).

# STAGED AUTONOMY — never ship full-auto on day one. Gate ACTIONS by risk + confidence,
# and roll the system out in stages: draft-only → execute-with-approval → auto-under-limits.
def gated(action, stage, confidence):
    if stage == "draft":                        # 1: the agent proposes, a human acts
        return propose(action)
    if stage == "approval":                     # 2: a human approves each action
        return execute(action) if human_approves(action) else skip(action)
    if stage == "auto":                         # 3: automatic — but only when it's safe
        if action.is_high_risk or confidence < 0.9:
            return require_approval(action)     # ← high-stakes STILL escalates (human-on-the-loop)
        return execute(action)
# Wrap the rollout in: a 1% canary (not 100%), a kill switch, a tested <5-min rollback,
# and CONTINUOUS evals that can halt a release on quality drift — not a one-time offline check.

Then the operational basics that turn a passing eval into a system you can leave running: a canary rollout (1%, not 100%), a kill switch, a tested rollback (minutes, not hours), blast-radius controls and feature flags per agent surface, and continuous evals wired to halt a bad release. The discipline is the same as the whole section: earn autonomy gradually, and never give an agent more reach than your guardrails can contain.

🧪 Try It Yourself

Reason these through, then check with the design walkthrough:

A multi-agent system keeps producing confident but wrong answers, and your base model is already the best available. What does MAST suggest the real problem is — and the single highest-leverage fix?
Walk the design sequence for a quick task: "summarize this one PDF." Where do you stop, and why?
Your agents sometimes stop early and sometimes loop forever. Which MAST category is this, and what two things does every loop need?
A teammate ships a brand-new agent straight to 100% auto because the offline eval was green. Name two things wrong and the safer rollout.
Anthropic's early system spawned 50 subagents for a simple query. What design fix prevents this?

→ (1) MAST says most failures are design, not model — a better model won't help; add verification (a critic/judge agent on critical outputs, with thresholds + retry limits) — the single biggest reliability lever, aimed at the ~21% verification-failure bucket. (2) Stop at step 1 (single agent). One small, sequential task isn't high-value/parallel/independent — multi-agent would just add coordination cost for no gain (L143). (3) Specification & design issues (missing termination condition). Every loop needs a hard cap (max steps/tokens) and a clear done signal (plus a no-progress detector). (4) No canary (ship 1%, not 100%) and no staged autonomy / kill switch (offline-green ≠ production-safe; evals should be continuous and able to halt the release). Roll out draft → approval → auto-under-thresholds. (5) An explicit effort-scaling rule in the orchestrator's prompt (e.g. simple fact-find → 1 agent, 3–10 calls) — agents can't judge effort themselves, so you encode it.

Mental-Model Corrections

"A better model will fix my flaky multi-agent system." Usually no — MAST shows most failures are design (spec, coordination, verification). Fix the structure: roles, termination, a verifier.
"Designing is choosing a framework." Framework comes last. Design is the decision sequence — single-or-multi, topology, coordination, boundaries, cost, reliability — then you pick a framework to express it.
"Add a verifier later if there are problems." Verification is the single biggest reliability lever and the most-skipped — design the critic and the termination cap in from the start.
"It's non-deterministic, so I can't really debug it." You debug it with tracing — capture every agent/call/handoff. No traces = no debugging, no evaluation, no shipping.
"Green offline eval means ship it." The deploy is where systems break. Use staged autonomy (draft → approval → auto), canaries, kill switches, and continuous evals.
"More agents = more capable." More agents = more coordination, cost, and failure modes (L144, L147 — Coordination & Cost). The best design uses the fewest agents that solve the problem, each clearly briefed and bounded.

Key Takeaways

A great multi-agent system is mostly great design. MAST (1,600+ traces) found most failures are design, not model — spec/design (~42%), inter-agent misalignment (~37%), verification (~21%) — and better base models won't fix them.
Design is a decision sequence: single-or-multi (L143) → decompose & roles → topology (L144) → coordination (L145) → boundaries / A2A (L146) → cost & controls (L147) → reliability & deploy. Walk it in order; most tasks stop at single.
Design for reliability: a verifier/critic agent on critical outputs (the biggest lever), explicit termination (a hard cap and a done signal — agents stop early or spin forever), and guardrails (limits, validation, scoped budgets). Decomposition is itself a guardrail.
Field lessons: think like your agents (simulate + watch), teach the orchestrator to delegate (briefs, not vibes), scale effort explicitly (no 50 subagents for a one-liner), and encode expert heuristics.
Observe & evaluate: agents are non-deterministic → trace everything; evaluate with humans + LLM-judges on versioned sets; make evals continuous and able to gate a release.
Ship in stages: draft → approval → auto-under-thresholds, with canaries, kill switches, rollback, and gated high-stakes actions. Earn autonomy gradually.
✅ Section 7 — Multi-Agent Systems — complete: why multi (L143) → supervisor/hierarchical (L144) → handoffs & shared state (L145) → A2A protocol (L146) → coordination & cost (L147) → designing a system (L148). You can now decide whether, how, and how much, and build it to ship.
Next — The Framework Landscape (LangGraph, CrewAI, smolagents, OpenAI Agents SDK, LlamaIndex): the tools that turn these designs into code.