Handling Tool Errors & Retries

Introduction

Every lesson so far assumed the happy path: the tool runs, returns a result, the agent continues. Production is not the happy path. APIs rate-limit you, networks drop, the model passes a bad argument, a tool throws, a sandbox times out. Studies find agents retry tool calls 15–30% of the time. Errors aren't the exception — they're the common case.

And here's the uncomfortable truth about agent reliability:

"The production harness accounts for ~98% of agent reliability" — the validation, retry, and recovery logic lives in your orchestration code, not in a cleverer prompt.

Error handling has two goals, and you must hit both:

Don't crash. One failing tool call must not take down the whole agent loop.
Let the agent decide. Give the model (or your code) enough information to recover — retry, fix the input, switch tools, or give up gracefully.

In this lesson:

The first rule — never drop an error (is_error), never swallow it
Error messages are prompts — make them actionable
Two failure kinds, two retry layers — transient (code backoff) vs semantic (the model adapts)
The idempotency trap, circuit breakers & graceful degradation, and the retry-spiral pitfalls

Scope: this builds on the L121 (Tools) tool lifecycle, L123 (Parallel & Sequential Tool Calls) parallel calls, and L124 (Code Execution & Sandboxing)'s sandbox (whose code also fails). It's the reliability capstone before L126 (Hands-On: Web Search + DB + Calculator)'s hands-on build.

An infographic titled 'Handling Tool Errors & Retries'. Tools fail constantly, and the gap between a demo agent and a production one is almost entirely error handling — the surrounding harness accounts for about 98 percent of agent reliability, not the prompt. THE FIRST RULE: never let a tool crash the loop and never silently swallow an error. Catch it and return a tool_result with is_error true, so the model SEES the failure and can decide what to do next — the way it reads any other observation. Swallowing an error (returning an empty string) is the worst case: the model thinks it succeeded and hallucinates a confident wrong answer. ERROR MESSAGES ARE PROMPTS: a vague 'Error' is useless; an actionable message with an error type and a fix ('InvalidCity: not found; use City, ST like Springfield, IL') lets the model self-correct, usually on the first retry. TWO KINDS OF FAILURE, TWO LAYERS OF RETRY: a TRANSIENT infrastructure error (HTTP 429 rate limit, a 5xx, a timeout, a dropped connection) is the harness's job — retry it in code with EXPONENTIAL BACKOFF plus JITTER (wait about 1s, 2s, 4s) so you don't create a thundering herd; the Anthropic SDK already auto-retries 429, 5xx, 408, 409 and connection errors. A SEMANTIC error (bad arguments, a 404, the wrong tool, a code traceback) won't fix itself by waiting — return is_error and let the MODEL adapt: fix the arguments, pick another tool, or ask the user. THE IDEMPOTENCY TRAP: a timeout is ambiguous — did the write go through? Blindly retrying a NON-IDEMPOTENT action like charge_card or send_email can double-charge or double-send. Attach an idempotency key so the server deduplicates and the retry is safe. PERSISTENT FAILURE: a circuit breaker (CLOSED, then OPEN after consecutive failures, then HALF-OPEN after a cooldown) stops hammering a dead dependency and triggers GRACEFUL DEGRADATION — fall back to a cached value, an alternative tool, a partial answer, or human escalation. PITFALLS: infinite retry spirals where the model keeps calling a broken tool (cap the turns and detect repeated identical failures), retrying writes, masking real bugs, and vague messages. THE LAYERED RULE: transient errors retry in code; semantic errors go back to the model; non-idempotent writes need an idempotency key; persistent failures trip a breaker and degrade gracefully. Takeaway: in production, errors are the common case — never crash, never swallow, feed actionable errors back to the model, back off transient failures, and never blind-retry a write.

The First Rule: Never Drop an Error

When a tool fails, you have three options, and two of them are wrong. You can let the exception propagate (the agent crashes), you can catch it and return something fake like "" (you swallow it), or you can catch it and tell the model. Only the third is correct.

The mechanism is the is_error flag on a tool_result:

for tool in tool_uses:
    try:
        result = execute(tool.name, tool.input)
        results.append({"type": "tool_result", "tool_use_id": tool.id,
                        "content": str(result)})
    except Exception as e:
        # DON'T crash the loop. DON'T swallow it. Hand the error BACK to the model.
        results.append({"type": "tool_result", "tool_use_id": tool.id,
                        "is_error": True,
                        "content": f"{type(e).__name__}: {e}"})    # actionable text
messages.append({"role": "user", "content": results})
# The model reads is_error like a ReAct observation (L111) and tries again.
# (In a parallel batch (L123), a failed call STILL needs its own is_error result.)

Two anti-patterns this avoids:

Crashing — an unhandled exception in one tool ends the entire run. A single flaky API shouldn't be fatal.
Swallowing — the worse one. If you return "" on failure, the model thinks the call succeeded and hallucinates over the gap: "The weather is sunny and 75°F!" — confidently wrong. A visible error is recoverable; an invisible one is a silent lie.

Remember is_error from L123 (Parallel & Sequential Tool Calls)'s parallel batches: every tool_use needs a matching tool_result, and a failed one is still a tool_result — just with is_error: true. Never simply omit it.

Error Messages Are Prompts

Once you've decided to hand the error back, what you say is everything. The model treats the error like a ReAct observation (L111) — it reads it and reasons about what to do. So an error message is a prompt, exactly like a tool description was in L122 (Designing Robust Tool Interfaces). Write it for the model:

# ❌ vague — the model has nothing to act on
{"type": "tool_result", "tool_use_id": id, "is_error": True, "content": "Error"}

# ✅ actionable — an Error Type + exactly how to fix it. An error message is a PROMPT.
{"type": "tool_result", "tool_use_id": id, "is_error": True,
 "content": "InvalidCity: 'Springfield, ZZ' not found. "
            "Provide a US city as 'City, ST' — e.g. 'Springfield, IL'."}

A good error message has two parts: an error type (InvalidCity, RateLimited, Timeout — a label the model can pattern-match) and a descriptive, actionable explanation (what went wrong and how to fix it). The payoff is large and well-documented: giving the model one correction attempt with a clear message resolves most argument/format errors on the first retry.

Compare the two outcomes: "Error" leaves the model guessing (it'll often retry the exact same failing call). "InvalidCity: … use 'City, ST'" tells it precisely what to change — and it does. Vague errors cause retry loops; actionable errors cause recovery.

Two Kinds of Failure, Two Layers of Retry

This is the core mental model of the whole lesson. Not all errors are retried the same way — because not all errors are the same kind.

	Transient (infrastructure)	Semantic (logical)
Examples	429 rate limit, 5xx, timeout, dropped connection	bad arguments, 404 not found, wrong tool, a code traceback
Will waiting fix it?	Yes — the service recovers	No — the same call fails forever
Who retries?	Your code — silently, with backoff	The model — by changing the call
Mechanism	exponential backoff + jitter	return `is_error`, model adapts

So there are two layers of retry, and matching them to the error kind is the skill:

Code-level retry handles transient errors before the model ever sees them — a 429 isn't the model's problem, it's the network's. Back off and try again.
Agent-level retry handles semantic errors — the model reads is_error and fixes the call (new arguments, a different tool, or asks the user).

The classic mistake is using the wrong layer: auto-retrying a semantic error (waiting 4s won't make "Springfield, ZZ" valid) or bouncing a transient error to the model (now it's reasoning about HTTP 429 instead of the task). Route by kind.

Code-Level Retry: Exponential Backoff + Jitter

For transient errors, the industry-standard mechanism is exponential backoff with jitter — wait longer after each failure (1s, 2s, 4s…), with a little randomness so simultaneous clients don't all retry in lockstep:

import time, random

def with_retry(fn, *, retries=4, base=1.0):
    for attempt in range(retries):
        try:
            return fn()
        except (RateLimitError, TimeoutError, ServerError):   # TRANSIENT only
            if attempt == retries - 1:
                raise
            delay = base * (2 ** attempt) + random.uniform(0, 0.5)  # backoff + JITTER
            time.sleep(delay)                                        # ~1s, 2s, 4s …

# The Anthropic SDK already retries 429 / 5xx / 408 / 409 / connection errors
# (max_retries defaults to 2). Wrap YOUR tools the same way — but NEVER auto-retry a
# semantic error: bad arguments won't fix themselves by waiting.

Why these specifics:

Exponential (not fixed) — immediate retries against a struggling service create a thundering herd that keeps it down. Backing off gives it room to recover.
Jitter — without randomness, all your clients retry at the same instants and synchronize into retry storms. Jitter spreads the load.
A max-retries cap — give up eventually (then escalate / degrade). Infinite retries are their own outage.

You often get the first layer for free: the Anthropic SDK auto-retries connection errors, 408, 409, 429, and ≥500 with backoff (max_retries defaults to 2). Apply the same pattern to your tools' transient calls — and only the transient ones.

The Idempotency Trap

Here is where retries get dangerous, and it ties straight back to the action tools of L121 (Tools) and "reads parallel, writes serial" from L123 (Parallel & Sequential Tool Calls). A retry is safe only if the operation is idempotent — i.e. running it twice has the same effect as running it once.

Reads are naturally idempotent (get_weather twice is fine). Writes usually aren't: charge_card, send_email, append_to_log, create_order. And the trigger is insidious — a timeout is ambiguous: did the write go through before the connection dropped, or not? If you blind-retry, you might do it twice:

# A timeout is AMBIGUOUS — did the charge go through or not?
charge_card(amount=50)          # timeout
charge_card(amount=50)          # blind retry → maybe a SECOND charge on top of a silent first  💸💸

# ✅ An idempotency key makes the retry safe — the server deduplicates on it.
key = "order-8821-charge"       # stable, derived from the operation
charge_card(amount=50, idempotency_key=key)   # timeout
charge_card(amount=50, idempotency_key=key)   # server sees the key → charges ONCE  ✓

The fix is an idempotency key: a stable identifier you attach to the operation so the server can recognize a duplicate and apply it once. (order-8821-charge, derived from the business operation — not a random value per attempt.) Most serious payment, messaging, and write APIs support one for exactly this reason.

The rule that prevents disasters: never blind-retry a non-idempotent write. Either make it idempotent (a key), or don't auto-retry it at all — surface the ambiguous result and let a human or the model decide. A double-charged customer is far worse than a failed request.

See It — The Error Recovery Lab

Below, a tool call fails. Pick the error class and flip the handling between Naive and Robust — and watch the outcomes split apart:

A tool call failed — pick the error CLASS (a transient 429 rate-limit, bad arguments, a crash/traceback, or a non-idempotent charge_card timeout) and flip the handling between ✗ Naive and ✓ Robust. Watch the outcomes diverge: immediate retries hammer a struggling API while exponential backoff rescues it; swallowing a bad-args error makes the model hallucinate a confident wrong answer while an is_error message lets it fix the call; an unhandled exception kills the whole loop while a returned traceback lets the model self-heal; and a blind retry on the charge double-charges the customer while an idempotency key keeps it to one. The lesson lands by hand: match the recovery strategy to the error class.

The four scenarios are the whole lesson in miniature:

429 rate limit (transient) → backoff recovers; immediate retry hammers the API.
Bad arguments (semantic) → is_error lets the model fix the city; swallowing it produces a confident wrong answer.
Tool crashed (semantic) → returning the traceback lets the model self-heal; an unhandled exception kills the loop.
charge_card timeout (non-idempotent) → an idempotency key keeps it to one charge; a blind retry double-charges.

Persistent Failure: Circuit Breakers & Graceful Degradation

Backoff handles a blip. But what if a dependency is down for minutes? Retrying — even with backoff — just burns time, tokens, and API credits while the whole pipeline stalls. The answer is a circuit breaker: after N consecutive failures, stop calling and fail fast.

It's a little state machine: CLOSED (normal) → too many failures → OPEN (stop calling, immediately fall back) → after a cooldown → HALF-OPEN (let one trial through) → success → CLOSED again.

# Persistent failure: stop hammering a dead dependency.
#   CLOSED --(failures > threshold)--> OPEN --(cooldown)--> HALF-OPEN --(ok)--> CLOSED
if breaker.is_open("flights_api"):
    return fallback()        # cached data, an alternative tool, a partial answer,
                             # or escalate to a human — i.e. degrade GRACEFULLY.

When the breaker is open, you degrade gracefully instead of failing hard. Graceful degradation is a philosophy more than a trick: expect failure, contain its blast radius, and preserve core functionality. Concretely:

Fallback tool — a second provider, or a cheaper/cached path.
Cached / stale data — "last known" beats nothing.
Partial answer — return what did work and say what didn't.
Human escalation — for unrecoverable or high-stakes failures, hand off.

The layered playbook: backoff for transient blips → circuit breaker for persistent outages → fallback for unavailability → human escalation for the unrecoverable.

🧪 Try It Yourself

Reason through these, then use the lab to confirm:

Predict: in the lab, set Bad arguments → Naive (swallow). What does the model output, and why is that the most dangerous failure mode?
A tool returns HTTP 429. Which retry layer handles it, and what exactly do you wait between attempts?
Your agent calls send_email, gets a timeout, and your harness auto-retries it. What's the risk, and the fix?
The model keeps calling a tool that returns "Error" over and over, never making progress. Name the root cause and two fixes.
flights_api has been failing for 3 minutes straight and your agent keeps retrying with backoff. What pattern are you missing, and what should happen instead?

→ (1) It hallucinates a confident, plausible answer (e.g. invents the weather) because a swallowed error looks like success — the most dangerous mode since it's a silent wrong answer, not a visible failure. (2) The code/harness layer, with exponential backoff + jitter (~1s, 2s, 4s…) — never bounce a 429 to the model. (3) A timeout is ambiguous, so a blind retry may double-send the email (non-idempotent write). Fix: an idempotency key so the server dedupes — or don't auto-retry the write. (4) The error message is vague ("Error"), so the model retries the same failing call — a retry spiral. Fixes: make the message actionable (error type + how to fix), and cap retries / detect repeated identical failures. (5) A circuit breaker — after N consecutive failures it should trip OPEN, stop calling, and degrade gracefully (fallback/cached/partial/escalate) instead of retrying a dead dependency.

Mental-Model Corrections

"If a tool fails, the agent should crash." No — catch it and return is_error. One failed call must not end the run.
"Just return empty/None on failure." That's the worst option — the model thinks it worked and hallucinates. Surface the error.
"is_error: true is enough." The flag lets the model know; the message lets it recover. Write an actionable message — it's a prompt.
"Retry everything the same way." Transient → back off in code; semantic → return to the model. Waiting never fixes bad arguments; bouncing a 429 to the model wastes a turn.
"Retries are always safe." Only for idempotent ops. Blind-retrying a write (charge/send) can double-execute — use an idempotency key.
"More retries = more reliable." Unbounded retries are an outage. Cap them, add a circuit breaker, then degrade gracefully.
"Reliability comes from a better prompt." It comes from the harness — validation, retry, backoff, breakers, fallbacks (the ~98%). The prompt is the small part.

Key Takeaways

Errors are the common case in production (agents retry 15–30% of calls). The harness — not the prompt — is ~98% of reliability.
Never crash, never swallow: catch failures and return a tool_result with is_error: true so the model can recover (and every parallel call still needs its own result).
Error messages are prompts: an error type + actionable fix resolves most mistakes on the first retry; "Error" causes retry loops.
Two failure kinds, two retry layers: transient (429/5xx/timeout) → code-level exponential backoff + jitter (the SDK does this for you); semantic (bad args/404/traceback) → return is_error, the model adapts. Route by kind.
The idempotency trap: a timeout is ambiguous — never blind-retry a non-idempotent write (double-charge!). Use an idempotency key.
Persistent failure: a circuit breaker (CLOSED→OPEN→HALF-OPEN) stops hammering a dead dependency; then degrade gracefully — fallback, cache, partial answer, or human escalation.
Avoid the retry spiral: cap retries, detect repeated identical failures, and never mask a real bug behind a retry.
Next — L126: Hands-On — build a real agent wiring together web search + a database + a calculator, applying everything from L121–L125 (Tools → Handling Tool Errors & Retries).