Self-Reflection & Self-Critique

Introduction

You've taught your agent to decompose (L127) and to choose an execution architecture (L128). This lesson is about the move that feels like it should make any agent smarter: letting it check and fix its own work. Generate an answer, critique it, revise. Self-reflection.

It's seductive — and it's the most over-trusted technique in the agent toolkit. The promise is real: Reflexion hit 91% on HumanEval by reflecting on failures. But so is the trap: a landmark DeepMind paper is titled, flatly, "Large Language Models Cannot Self-Correct Reasoning Yet." Both are true. The whole skill is knowing which situation you're in.

Self-reflection without an external signal is opinion about opinion. Grounded in a real check, it's a superpower. The difference is everything.

In this lesson:

The reflection loop (generate → critique → revise) and the techniques — Reflexion, Self-Refine, CRITIC, CoVe
The hard truth — why pure self-correction of reasoning can make answers worse
When reflection helps, and how to ground it so it reliably does
How it relates to evaluator-optimizer (L120), the memory angle, and the 2026 reality

Scope: this is self-critique. Tree-of-Thought / search (exploring many reasoning paths) is L130 (Tree-of-Thought & Search-Based Reasoning); re-planning & recovery (fixing a broken plan, not an answer) is L131 (Replanning & Recovering From Failure).

An infographic titled 'Self-Reflection & Self-Critique'. Self-reflection is the agent examining its own output, critiquing it, and revising: a generate -> self-critique -> revise loop, the verbal reinforcement family of techniques. THE TECHNIQUES: Reflexion (Shinn 2023) has the agent write a natural-language post-mortem of a FAILURE signal and store it in episodic memory before retrying, reaching 91 percent pass-at-1 on HumanEval; Self-Refine (Madaan 2023) uses one model with three prompts (generate, feedback, refine); CRITIC verifies with external tools before critiquing; and Chain-of-Verification (CoVe) turns 'is this right?' into independent verification questions. THE HARD TRUTH (Huang et al. 2023, 'LLMs Cannot Self-Correct Reasoning Yet'): INTRINSIC self-correction, where a model fixes its answer using only its own judgment with NO external feedback, often does NOT improve reasoning and can actually DEGRADE it — the model second-guesses and 'corrects' a correct answer into a wrong one; earlier positive results often leaked an oracle that told the model when to stop. The model also suffers self-preference bias, over-rating its own work. SO WHEN DOES REFLECTION HELP? Two cases. First, when it is GROUNDED in a real external signal — run the tests, call a tool, search, check ground truth (CRITIC, CoVe, Reflexion's reward) — the critique is anchored in fact, not opinion, and reliably fixes errors. Second, for OPEN-ENDED tasks (writing, completeness, structure, safety) where there is no single right answer to overwrite, so self-critique only adds coverage. It HURTS for pure reasoning or math introspection (over-editing, flipping right to wrong), and it always costs extra passes. REFLECTION vs EVALUATOR-OPTIMIZER (L120): the evaluator-optimizer pattern used a SEPARATE evaluator; reflection is often the SAME model critiquing itself (intrinsic), and Reflexion adds cross-attempt episodic MEMORY. THE 2026 NUANCE: reasoning models already self-check internally during extended thinking, so stacking an external intrinsic-reflection loop on top has diminishing returns — the value is in GROUNDING (a verifier) or cross-attempt memory, not more introspection. Takeaway: generate, critique, revise — but never trust ungrounded self-correction on reasoning; ground the critique in tests, tools, or verification, and reserve pure self-critique for open-ended work.

The Reflection Loop

At its core, reflection is a three-step loop — and you've seen its shape before, in the self-correction chain (L116) and evaluator-optimizer (L120):

generate → critique → revise

The simplest form is Self-Refine (Madaan et al., 2023): one model, three prompts — draft it, critique the draft, rewrite using the critique:

# Self-Refine (Madaan 2023): generate → self-feedback → refine, with one model.
draft    = model(task)
critique = model(f"Critique this draft. List SPECIFIC issues and how to fix each.\n{draft}")
revised  = model(f"Task:\n{task}\nDraft:\n{draft}\nIssues:\n{critique}\n"
                 f"Rewrite it, fixing every issue.")

# Cheap, and genuinely useful for OPEN-ENDED work (writing, completeness, structure).
# But on a REASONING task, this exact loop can make a CORRECT answer worse — the model
# second-guesses itself with no real signal (Huang et al. 2023). Read on.

This whole family is called verbal reinforcement learning: the agent improves within inference by reasoning about its errors in natural language — no gradient updates, no training. It's cheap and, for the right tasks, effective.

The catch is hiding in plain sight in that code: the critique comes from the same model that wrote the draft. When there's a real signal to critique against, that's fine. When there isn't — when the model is just asked "is this right?" — you're about to see why it's dangerous.

The Techniques

Four named techniques cover the landscape — they differ mainly in what the critique is anchored to:

Self-Refine (Madaan 2023) — self-feedback only (intrinsic). Generate → feedback → refine, iteratively. The baseline; ~20% average lift on the tasks where it works.
Reflexion (Shinn 2023) — reflect on a failure signal (a reward, a failed test), write a verbal post-mortem, store it in episodic memory, and retry. 91% pass@1 on HumanEval. The reflection is anchored to executed tests, not introspection.
CRITIC (Gou 2024) — tool-interactive self-correction. Before critiquing, verify with a tool (run code, search, calculate) and critique against the result.
Chain-of-Verification (CoVe) (2024) — generate, then write verification questions, answer them independently, and revise. Designed to cut hallucination.

Notice the pattern: the reliable techniques (Reflexion, CRITIC, CoVe) all inject an external or independent signal. The purely intrinsic one (Self-Refine alone) is the one with the asterisk. That asterisk is the next section.

The Hard Truth — LLMs Can't Self-Correct Reasoning (Yet)

This is the section that separates a world-class engineer from someone who just read the Reflexion abstract. In "Large Language Models Cannot Self-Correct Reasoning Yet" (Huang et al., DeepMind, ICLR 2024), the authors test intrinsic self-correction — a model fixing its answer using only its own judgment, with no external feedback — and find:

On reasoning tasks, intrinsic self-correction often fails to improve performance, and frequently makes it worse.

Why does "think again" hurt?

The model can't reliably tell its right answers from its wrong ones. If it could, it would have answered correctly the first time. Asked to re-examine, it's as likely to "fix" a correct answer as a wrong one.
Over-editing / second-guessing. Prompted to find fault, it finds fault — even where none exists — and edits a good answer into a bad one. (The widget's "is 17 prime?" case: Yes → second-guessed → "not prime.")
Self-preference bias (from L120 — Evaluator-Optimizer). A model over-rates output that reads like its own — so its self-critique is soft where it should be harsh.
The oracle leak. Many earlier "self-correction works!" results quietly used an oracle — ground-truth labels that told the model when to stop correcting. Remove the oracle (true intrinsic setting) and the gains evaporate.

The honest summary: a model is not a reliable judge of its own reasoning. "Reflect harder" is not a fix for a wrong proof or a bad calculation — it's a coin flip that sometimes breaks a right one.

See It — Reflection That Helps, Misses, and Degrades

Feel the whole truth in one widget. Pick a draft and flip between self-critique only and + external check:

Pick a draft — buggy code, an already-correct reasoning answer, or a weak essay — and flip the reflection SOURCE between 'self-critique only' (intrinsic) and '+ external check' (grounded). The outcomes tell the honest story: intrinsic self-critique MISSES its own code bug (overconfident, shares its blind spot) and DEGRADES the correct reasoning answer (it second-guesses 17-is-prime into 'not prime' — the Huang et al. 2023 finding), yet genuinely IMPROVES the open-ended essay (completeness has no single right answer to overwrite). Switch to the grounded check — run the tests, verify the math — and reflection becomes reliable everywhere. The lab makes the rule physical: ground your reflection, and never trust ungrounded self-correction on reasoning.

The three drafts map exactly to the research:

Buggy code — intrinsic critique misses the bug (it's overconfident, sharing its own blind spot); the grounded check (run the tests) catches and fixes it.
Correct reasoning — intrinsic critique degrades it (second-guesses a right answer into a wrong one — Huang et al.); the grounded check holds the correct answer.
Weak essay — intrinsic critique improves it: open-ended writing has no single right answer to overwrite, so more self-scrutiny just adds coverage.

Same technique, three different verdicts — driven entirely by task type and whether the critique is grounded.

When Reflection Helps — and How to Ground It

So when should you reach for reflection? Two clear green-lights, and a clear red-light:

✅ Ground it in an external signal. This is the #1 move: replace "are you sure?" with "here is what the test/tool/search says." Reflexion (a reward/tests), CRITIC (tools), CoVe (independent verification) all do this. A grounded critique is anchored in fact, not opinion — and it reliably fixes errors:

# The fix: don't ask "are you sure?" — give the critique a REAL signal.

# CRITIC (Gou 2024): verify with a TOOL, then critique against the actual result.
result = run_tests(draft_code)              # or: search(claim) · calc(expr) · compile(...)
if result.failed:
    revised = model(f"{draft_code}\n\nThe tests failed:\n{result.log}\nFix the cause.")

# Chain-of-Verification (CoVe): turn "is this right?" into checkable sub-questions,
# answered INDEPENDENTLY (so the model can't just re-confirm its own claim).
questions = model(f"List verification questions for this answer:\n{draft}")
answers   = [verify(q) for q in questions]  # answer each on its own
revised   = model(f"{draft}\n\nChecks:\n{answers}\nRevise anything a check contradicts.")

✅ Use pure self-critique for open-ended work. Writing, summaries, completeness, tone, structure, safety/policy checks — tasks with no single correct answer to overwrite. Here self-reflection genuinely helps (Self-Refine's home turf), because the worst case is "different," not "wrong."

❌ Don't trust ungrounded self-correction on reasoning/math/facts. This is where it degrades. If you can't ground it, often the best move is don't reflect — a single confident answer beats a second-guessed one.

Two more practical guards: cap the loop (reflection has diminishing returns and a cost — like L120 — Evaluator-Optimizer, gains saturate in 1–2 rounds), and prompt for a specific, harsh critique ("list concrete issues + fixes," not "improve this") so a pass that does happen is worth its tokens.

Reflection vs Evaluator-Optimizer — and the Memory Angle

Two connections sharpen the concept.

vs Evaluator-Optimizer (L120). They're the same loop family (Self-Refine/Reflexion roots), but:

Evaluator-optimizer (L120) is a workflow pattern with a separate evaluator — ideally a different/stronger model judging the generator's output. The separation is the point: it dodges self-preference.
Self-reflection here is usually the same model critiquing itself (intrinsic) — which is exactly why the Huang et al. warning bites harder. When you can use a separate evaluator or a real verifier, do — it's strictly more reliable than talking to yourself.

The memory angle. Reflexion's real innovation isn't "critique" — it's persisting the critique. It writes each failure's lesson into an episodic memory that's fed into the next attempt, so the agent doesn't repeat the mistake. That's reflection as learning across attempts, not just editing one answer — and it's why Reflexion needs multiple tries with a signal, not a single introspective pass. (Memory gets its own treatment later in the course.)

The 2026 Nuance — Reasoning Models Already Reflect

Here's what's changed under everyone's feet. Modern reasoning models self-critique internally during extended thinking — a long chain of thought is a generate-check-revise process happening inside one turn (the planning point from L127 — Task Decomposition & Planning, the test-time-compute idea from L120 — Evaluator-Optimizer). They catch many of their own slips before you ever see the output.

Two consequences for explicit reflection loops:

Diminishing returns on intrinsic reflection. Bolting an external "now critique yourself" pass onto a model that already reasoned hard often adds latency and cost for little gain — and can even re-introduce the over-editing problem.
The value moved to grounding and memory. Where explicit reflection still clearly pays off in 2026 is exactly where the model can't do it alone: a real external verifier (run the tests, hit a tool, check a source) and cross-attempt memory (Reflexion). Those add information the model doesn't have; another round of introspection doesn't.

The rule that survives every model upgrade: reflection adds value when it adds information — a test result, a tool's answer, a remembered lesson. It adds risk when it just adds opinion.

🧪 Try It Yourself

Reason through these, then use the lab to confirm:

Predict: in the lab, set Correct reasoning → self-critique only. What happens to the right answer, and what's the name of this finding?
Your agent solves math problems and you add a "double-check your answer" reflection step. Accuracy drops. Why — and what should you add instead?
For which of these is pure (ungrounded) self-critique safest: (a) a SQL query's correctness, (b) the tone of a customer email, (c) a physics calculation? Why?
What single change turns Self-Refine (unreliable on code) into CRITIC/Reflexion (reliable on code)?
Why is a separate, stronger evaluator (L120) generally better than a model reflecting on itself?

→ (1) It often degrades — the model second-guesses and "corrects" the correct answer into a wrong one. This is Huang et al. 2023, "LLMs cannot self-correct reasoning yet" (intrinsic self-correction). (2) A model can't reliably judge its own reasoning, so ungrounded "double-check" introduces errors. Add a real verifier — run a calculator/code/check — and critique against that (CRITIC), not its own opinion. (3) (b) the email tone — it's open-ended with no single right answer to overwrite, so self-critique only adds coverage. (a) and (c) are verifiable reasoning, where ungrounded reflection can degrade — ground them with a DB/execution or a calculation check. (4) Ground the critique in an external signal — run the tests and critique against the failure log (instead of asking the model if it's sure). (5) A separate/stronger evaluator avoids self-preference bias and doesn't share the generator's blind spots, so its critique carries real information rather than echoing the author.

Mental-Model Corrections

"Telling the model to reflect makes it smarter." Only sometimes. On reasoning, ungrounded self-correction often degrades the answer (Huang et al.). It's not a free upgrade.
"Reflexion proves self-correction works." Reflexion works because it reflects on a real failure signal (executed tests) and stores the lesson — not because the model introspects well.
"A model can check its own work." It can't reliably judge its own reasoning — if it could, it'd have been right the first time. Self-preference bias makes its self-critique too soft.
"More reflection rounds = better." Gains saturate after 1–2 rounds; extra rounds cost tokens and risk over-editing a good answer into a bad one. Cap it.
"Reflection and evaluator-optimizer are the same." Same loop family, but evaluator-optimizer (L120) uses a separate (ideally stronger) evaluator; self-reflection is often the same model — which is exactly why it's riskier.
"It works for code, so it works for math." It works for code when grounded in tests. Ungrounded, it's unreliable for both — code just happens to have an easy external verifier.
"Reasoning models removed the need to ground reflection." They reflect internally, so extra intrinsic reflection has diminishing returns — but grounding (a verifier) and memory still add information they lack.

Key Takeaways

Self-reflection = generate → critique → revise (verbal reinforcement: Self-Refine, Reflexion, CRITIC, CoVe). No training — improvement within inference.
The hard truth (Huang et al. 2023): intrinsic self-correction — no external feedback — often doesn't help reasoning and can degrade it (the model "corrects" correct answers; self-preference bias; the oracle leak).
Reflection helps in two cases: when grounded in a real signal (tests, tools, search, independent verification — Reflexion/CRITIC/CoVe), and for open-ended tasks (writing, completeness) with no right answer to overwrite.
Ground it: replace "are you sure?" with "here's the test/tool/verification result." That single move is the difference between reliable and risky. Cap the loop (gains saturate in 1–2 rounds).
vs L120: evaluator-optimizer uses a separate/stronger evaluator (dodges self-preference); reflection is often the same model — prefer a separate verifier when you can. Reflexion adds cross-attempt memory — reflection as learning, not just editing.
2026: reasoning models self-check internally, so extra intrinsic reflection has diminishing returns — value is in grounding and memory, the information the model lacks.
The rule: reflection adds value when it adds information, and adds risk when it just adds opinion.
Next — L130: Tree-of-Thought & Search-Based Reasoning — instead of revising one line of thought, explore and search over many.