Evaluator-Optimizer

Introduction

Pattern 5 of 5 (L115) — the last, and the only one that iterates. Anthropic: "In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop."

Every pattern so far has been feed-forward: chaining runs a fixed sequence once, routing picks one path, parallelization fans out, orchestrator-workers decomposes and synthesizes — all in a single pass. Evaluator-optimizer adds a feedback loop: a generator drafts, an evaluator critiques against criteria, and the generator revises with that critique — over and over — until the output clears a bar. As Anthropic puts it, it's "analogous to the iterative writing process a human writer might go through when producing a polished document." You already met a one-shot version of this — the self-correction chain (draft→review→refine) in L116 (Prompt Chaining). This makes that review step into an open-ended loop.

In this lesson:

The loop (generate → evaluate → refine) and Anthropic's exact when-to-use
Building it from scratch — and why the evaluator is an LLM-as-judge (with all its biases)
The stop condition people get wrong, the roots (Self-Refine, Reflexion, test-time compute)
The honest failure modes, when not to loop, and the framework (a cyclic LangGraph)

An infographic titled 'Evaluator-Optimizer'. One LLM generates a response while another provides evaluation and feedback in a loop — analogous to the iterative writing process a human writer might go through. THE LOOP: a GENERATOR drafts a response; an EVALUATOR scores it against a rubric (correctness, edge cases, style) and returns a verdict (PASS or NEEDS_IMPROVEMENT) plus specific, actionable feedback; if not PASS, the generator revises with that feedback and the loop repeats until it passes or hits a max-iterations cap. WHEN TO USE IT (Anthropic): it is particularly effective when you have clear evaluation criteria and when iterative refinement provides measurable value; the two signs of good fit are that LLM responses can be demonstrably improved when a human articulates feedback, and that the LLM can provide such feedback. Examples: literary translation where an evaluator LLM critiques nuance the first pass misses, and complex search needing multiple rounds where the evaluator decides whether further searches are warranted. THE STOP CONDITION IS CRITICAL: loop until perfect never halts because the judge keeps finding new nits (moving goalposts), so always bound the loop with a cap (3 to 5) and a budget; gains saturate after about one or two rounds. THE HONEST FAILURES: a same-model or same-family evaluator shares the generator's blind spots and rubber-stamps bugs (self-preference bias — an LLM rates outputs that read like its own style higher); diminishing returns and over-editing; reward hacking where the generator games the rubric. The strongest mitigation is an external ground-truth verifier when one exists (run the tests, a compiler) — Reflexion reached 91 percent pass at 1 on HumanEval versus about 80 percent baseline by looping on executed tests rather than self-judgment. It is the only one of the five patterns with an iterative generator-evaluator loop; prompt chaining draft-review-refine is a one-shot preview; it is the workflow embodiment of test-time compute. Roots: Self-Refine and Reflexion. Takeaway: generate, critique, refine — but only when you have clear criteria and iteration measurably helps; cap the loop, give specific actionable feedback not a bare score, and prefer a real verifier over an LLM judging its own work.

When To Use It — Anthropic's Two Signs

This pattern is powerful and easy to misapply, so memorize the gate. Anthropic's when-to-use, verbatim:

"This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. The two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback."

Unpack the two preconditions — both must hold:

Clear evaluation criteria. You can write down what "good" means (a rubric, tests, a spec). Without this, the evaluator has nothing objective to check, and the loop just churns.
Iteration provides measurable value. Revisions demonstrably improve the output — and the LLM can articulate the feedback that drives the improvement.

Anthropic's two examples (verbatim): "Literary translation where there are nuances that the translator LLM might not capture initially, but where an evaluator LLM can provide useful critiques" and "Complex search tasks that require multiple rounds of searching and analysis to gather comprehensive information, where the evaluator decides whether further searches are warranted."

If you can't articulate the criteria, you don't have an evaluator-optimizer task — you have a generator and a coin flip.

Build the Loop From Scratch

The whole pattern is a while loop with a structured verdict and a hard cap. The evaluator must return machine-readable output so the loop condition is code, not vibes:

import anthropic, json
client = anthropic.Anthropic()
MAX_ITERS = 4                         # a HARD cap — the loop MUST terminate

# The evaluator returns STRUCTURED output, so the stop condition is code, not vibes.
VERDICT = {
  "type": "object", "required": ["status", "scores", "feedback"],
  "properties": {
    "status":   {"type": "string", "enum": ["PASS", "NEEDS_IMPROVEMENT"]},
    "scores":   {"type": "object", "required": ["correctness", "edge_cases", "style"],
                 "properties": {"correctness": {"type": "integer"},
                                "edge_cases":  {"type": "integer"},
                                "style":       {"type": "integer"}}},
    "feedback": {"type": "string"}}}  # specific + ACTIONABLE, not just a number

def generate(task, draft=None, feedback=None):
    if draft is None:
        prompt = f"Task:\n{task}\n\nWrite the best solution you can."
    else:                              # revise WITH the prior draft + the critique
        prompt = (f"Task:\n{task}\n\nYour previous attempt:\n{draft}\n\n"
                  f"A reviewer found these issues:\n{feedback}\n\nRewrite it, fixing every issue.")
    return client.messages.create(model="claude-sonnet-4-5", max_tokens=1500,
        messages=[{"role": "user", "content": prompt}]).content[0].text

def evaluate(task, draft):             # a STRONGER, DIFFERENT model judges (see below)
    r = client.messages.create(model="claude-opus-4-8", max_tokens=800,
        output_format={"type": "json_schema", "schema": VERDICT},
        messages=[{"role": "user", "content": EVALUATOR_PROMPT.format(task=task, draft=draft)}])
    return json.loads(r.content[0].text)

def optimize(task):
    draft, prev = generate(task), -1
    for i in range(MAX_ITERS):
        v = evaluate(task, draft)
        if v["status"] == "PASS":                  # STOP: passed the bar
            return draft, i + 1
        total = sum(v["scores"].values())
        if total <= prev:                           # STOP: no improvement / plateau
            return draft, i + 1
        prev  = total
        draft = generate(task, draft, v["feedback"])# revise with the feedback
    return draft, MAX_ITERS                          # STOP: hit the cap
# Three real stops: PASS, plateau, and the cap. "Loop until perfect" is NOT a stop.

Two design choices already matter here:

The feedback is threaded back in. The generator doesn't re-derive from scratch — it gets its previous draft + the critique and patches. That's the engine of improvement.
There are three real stops — PASS, a score plateau, and the cap. Notice what's not a stop: "loop until it's perfect." We'll see why that never terminates.

The Evaluator Is an LLM-as-Judge (Mind Its Biases)

The evaluator is an instance of LLM-as-a-judge — and the research says that's legitimate: strong judges reach >80% agreement with human preferences (Zheng et al., MT-Bench, 2023), about the same as humans agree with each other. But a judge is only as good as how you prompt it:

EVALUATOR_PROMPT = """You are a strict, fair reviewer. Do NOT rewrite the response —
your only job is to evaluate it and tell the author exactly how to improve.

TASK:
{task}

RESPONSE TO EVALUATE:
{draft}

Reason BEFORE you score. Score each criterion 1-5:
- correctness:  every case is handled correctly
- edge_cases:   empty / malformed / boundary inputs
- style:        clear, idiomatic, well-named

status = "PASS" only if every criterion is >= 4, else "NEEDS_IMPROVEMENT".
In `feedback`, give a numbered list of CONCRETE, ACTIONABLE fixes — quote the exact
problem and state the change. No vague praise. (A PASS may leave feedback empty.)"""
# Three things make a judge reliable: reason-BEFORE-score (G-Eval), an EXPLICIT
# threshold so PASS isn't a vibe, and "don't rewrite" to keep the two roles separate.

Three reliability levers are baked into that prompt: reason before you score (the core of G-Eval), an explicit threshold (≥4 on all, so PASS isn't subjective), and "don't rewrite" (keep the evaluator from quietly becoming the generator).

Now the part that bites people — LLM judges are biased, with measured magnitudes:

Self-preference / self-enhancement. A model rates its own (or its family's) outputs higher — GPT-4 shows a self-preference score of ~0.52; the cause is familiarity (it favors low-perplexity text that reads like its own style), not true self-recognition. A generator grading itself over-passes.
Verbosity bias (judges prefer longer answers — >90% of the time in one study) and position bias (favoring an answer by its slot).

The single biggest mitigation: make the evaluator a different (ideally stronger) model, or at least a fresh context — so it isn't anchored on the generation and doesn't share the generator's blind spots. Judging quality tracks model capability, so a strong judge is genuinely better.

The Stop Condition (the Part People Get Wrong)

The reference loop in Anthropic's own cookbook is literally while True — no cap. That's fine for a demo and dangerous in production. The loop must terminate on something other than "perfect":

PASS — the evaluator's verdict clears the bar.
A max-iterations cap — typically 3–5.
A score plateau — stop when the aggregate score stops rising (or regresses).
A cost / wall-clock budget.

Why "loop until perfect" never halts: an LLM judge applies fuzzy, slightly non-deterministic criteria and will almost always surface some new nit each pass — the goalposts move. There's no fixed point where it declares perfection, and self-preference quietly inflates scores without real gains. Empirically, refinement saturates after ~1–2 rounds (Self-Refine), and over-editing past that can even degrade quality (drifting from intent, "regression to blandness"). Always pair a soft stop (PASS / plateau) with a hard stop (cap / budget). The widget's picky mode is exactly this failure — feel it.

See It Loop (and Watch It Fail)

Step the loop below on a real coding task — parse_duration("1h30m"). Watch the generator draft, the evaluator score it against a rubric and hand back feedback, and the next draft improve. Then flip the evaluator's mode to see all three failure modes:

Step the generate → evaluate → refine loop on a real code task (parse_duration). The generator drafts, the evaluator scores it against a rubric (correctness / edge cases / style) and returns PASS or NEEDS_IMPROVEMENT with actionable feedback, and the generator revises with that feedback. Then switch the evaluator's mode: 'rigorous' converges to a correct PASS; 'picky' invents a new nit every round and never passes (so it slams into the cap — the reason you MUST bound the loop); 'shares a blind spot' rubber-stamps a buggy version that the external test suite catches red. The whole pattern, including its three failure modes, in one widget.

Run all three evaluator modes:

Rigorous → it converges to a correct PASS in 3 iterations, each round threading specific, actionable feedback into the next draft. With run external tests on, the verifier agrees (green).
Picky (never satisfied) → the evaluator invents a new nit every round (add a docstring → shorten it → rename a variable). It never passes and slams into the cap — moving goalposts, the reason you must bound the loop.
Shares a blind spot → a same-family judge rubber-stamps a version with a real bug (it assumed h→m→s order, so "30m1h" mis-parses) — but the external test fails red. That's the whole argument for a verifier over an LLM judging itself.

The Roots — Self-Refine, Reflexion & Test-Time Compute

This isn't new; it's a well-studied family, and the numbers tell you when it's worth it.

Self-Refine (Madaan et al., 2023) — the same LLM does generate → feedback → refine with no extra training, just prompting; it lifted quality ~20% on average across 7 tasks. Evaluator-optimizer is essentially Self-Refine with the feedback step split into a separate (ideally different) evaluator — which removes the self-preference problem of a model grading itself.
Reflexion (Shinn et al., 2023) — an Actor generates, an Evaluator scores (for code, via self-generated unit tests), and a Self-Reflection model turns the result into verbal feedback stored in memory across attempts. It hit 91% pass@1 on HumanEval vs ~80% baseline — and the gain came from executing tests, not self-judgment. The authors found that blind retrying without the verbal reflection didn't help at all.
Test-time compute. This loop is inference-time scaling by hand — spend more compute at runtime (extra generate/evaluate passes) to raise quality without retraining. The catch for 2026: modern reasoning models already do a lot of this internally (long chains of self-checking), so an external evaluate-refine loop has diminishing marginal value on top of a strong reasoning model. Reach for the explicit loop when you need an independent evaluator (a different model, a rubric, or real tests) — not just more of the same model's introspection.

The Honest Failures (and When NOT to Loop)

Beyond never-terminating, the pattern has sharp failure modes:

Correlated blind spots. If the evaluator shares the generator's model/family, it can't catch the generator's class of error — and self-preference makes it pass lookalike output. (The widget's blind-spot mode.) Fix: a different/stronger judge, or an external verifier.
Diminishing returns / over-editing. Most gain is in the first 1–2 rounds; more loops cost money for little — and can degrade the output. Fix: stop on plateau; cap iterations.
Reward hacking (Goodhart's law). The generator learns to satisfy the rubric's literal checks — sycophancy, padding, "citation theater" — without truly improving. Fix: broaden/rotate the rubric; hold out hidden criteria; periodic human audits.
A weak/noisy evaluator is worse than none — it sends the generator chasing wrong feedback. Feedback quality is the bottleneck.

When NOT to use it: you can't articulate clear criteria (the gate fails); a single pass is already good enough; iteration doesn't measurably help; the path is latency/cost-sensitive; or the task is subjective with no ground truth, where the judge's verdict is just noise. And the strongest principle of all: when an external, ground-truth verifier exists — tests that run, a compiler, a type checker, a fact-checker — use that, not (or in addition to) an LLM judge. Self-judgment is a weak signal; execution is a strong one.

Then — and Only Then — the Framework

You built the loop with a for and an if, so the framework holds no mystery. In LangGraph it's a cyclic graph — a generator node, an evaluator node, and a conditional edge that loops back on NEEDS_IMPROVEMENT or routes to END on PASS:

# In LangGraph it's a CYCLIC graph: generator → evaluator → a conditional edge that
# loops BACK on NEEDS_IMPROVEMENT or routes to END on PASS.
from langgraph.graph import StateGraph, START, END

def route(state):
    if state["status"] == "PASS" or state["iters"] >= MAX_ITERS:   # <- YOUR cap, in state
        return END
    return "generator"

builder.add_edge(START, "generator")
builder.add_edge("generator", "evaluator")
builder.add_conditional_edges("evaluator", route, {"generator": "generator", END: END})
# Put the cap in STATE. Don't rely on LangGraph's recursion_limit (default 25) — it
# raises a HARD exception, so it's a safety net, not a graceful stop.

The one thing to get right: put the iteration cap in your state and check it in the router. Don't lean on LangGraph's recursion_limit (default 25) — it throws a hard exception when exceeded, so it's a crash-guard, not a graceful stop. A plain LangChain while loop with .with_structured_output(Verdict) on the judge works just as well when you don't need persistence or streaming. Build the loop once by hand; the framework just draws it as a graph. (Same lesson as L111 — Building a ReAct Agent From Scratch and L116–L119 — Prompt Chaining → Orchestrator-Worker.)

🧪 Try It Yourself

Reason through these, then use the widget to confirm:

Predict before clicking: set the evaluator to picky and step the loop. Does it ever PASS? What stops it, and what's the name for what the evaluator is doing?
Your generator and evaluator are the same model. You ship a feature the evaluator rated PASS — and it has a bug. Why did the judge miss it, and what are the two fixes?
You're building a code-generation loop. What evaluator should you use instead of an LLM judge whenever you can — and why is it stronger?
A teammate's evaluator-optimizer runs 8 iterations on every request and the outputs aren't noticeably better than iteration 2. What two things are wrong?
When is evaluator-optimizer the wrong pattern entirely — name two conditions from Anthropic's gate.

→ (1) No — it never passes; it hits the cap (here, 4). The evaluator is moving the goalposts (a new nit each round), which is exactly why "loop until perfect" never terminates and you must bound it. (2) The same-model judge shares the generator's blind spot and exhibits self-preference (it over-rates output that reads like its own style), so it rubber-stamped the bug. Fixes: (a) use a different/stronger model (or a fresh context) as the evaluator; (b) add an external verifier — run the tests. (3) An external, execution-based verifier (run the unit tests / compiler). It checks ground truth, not the model's opinion of itself — which is exactly how Reflexion beat the baseline (91% vs ~80% pass@1) by looping on executed tests. (4) (a) No cap/plateau stop — gains saturate after ~1–2 rounds, so iterations 3–8 are wasted money (and risk over-editing); (b) likely a weak signal — without an external verifier or a sharper rubric, extra loops can't find real improvements. (5) Any two of: no clear/articulable evaluation criteria; a single pass already meets the bar; iteration doesn't measurably help; or a subjective task with no ground truth (the judge is noise).

Mental-Model Corrections

"Loop until it's perfect." That never halts — the judge keeps finding nits. Cap it (3–5) and stop on PASS or a plateau. "Perfect" is not a stop condition.
"The generator can grade its own work." It over-passes — self-preference makes a model favor output that reads like its own. Use a different/stronger evaluator, or an external verifier.
"A score is enough feedback." A bare number tells the generator that it failed, never what to change. Feedback must be specific and actionable (what's wrong + how to fix).
"More iterations = better." Gains saturate after ~1–2 rounds, and over-editing can degrade quality. Most of the value is early.
"An LLM judge is the gold standard." When a real verifier exists (tests, a compiler, a checker), it's a far stronger signal than self-judgment — "LLMs can't reliably self-correct reasoning without external feedback."
"It's the same as prompt chaining's review step." Chaining's draft→review→refine is a one-shot preview; evaluator-optimizer is an open-ended loop that decides whether to iterate again.
"Use it for any quality-critical task." Only with clear criteria and measurable gains from iteration. No criteria → no evaluator → no pattern.

Key Takeaways

Evaluator-optimizer = a generator and an evaluator in a loop: generate → evaluate (PASS / NEEDS + feedback) → refine with the feedback → repeat. Pattern 5 of 5, and the only iterative one.
Use it only when (Anthropic) you have clear evaluation criteria and iteration provides measurable value — a human can articulate feedback and the LLM can give it.
Build it: a while loop, a structured verdict (so the stop is code), and feedback threaded back into the next draft.
The evaluator is an LLM-as-judge: give it a rubric + per-criterion scores, make it reason before scoring, and use a different/stronger model to dodge self-preference and shared blind spots.
The stop condition is the trap: "loop until perfect" never halts. Cap at 3–5, stop on plateau, set a budget. Gains saturate fast.
Roots & truth: Self-Refine (~20% avg), Reflexion (91% pass@1 via executed tests); it's test-time compute — and a strong reasoning model already does much of it internally. Prefer an external verifier over self-judgment whenever one exists.
Framework: a cyclic LangGraph (generator → evaluator → conditional loop-back), with your cap in state.
That completes the five workflow patterns (chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer). Next, Section 3 — Tools: giving agents hands (L121).