Agentic RAG: Corrective RAG (C-RAG) & Self-RAG

Introduction

Every RAG system you've built so far — even the production stack of L102 — shares one hidden assumption: it's a fixed, one-shot pipeline. Retrieve once, generate once, done. And that pipeline has a fatal blind spot: when retrieval returns garbage, it generates from garbage. It can't notice the documents are irrelevant, can't go look somewhere else, and can't catch itself hallucinating.

Agentic RAG removes that assumption. It turns RAG from a pipeline into a loop — an agent that retrieves, grades what it got, corrects course, generates, and then checks its own answer, iterating until it's confident (or hits a budget). This is the culmination of routing (L101) and iterative reasoning (L100): retrieval becomes a self-correcting reasoning process, and it's the bridge from RAG to true agents.

In this lesson:

Why standard RAG can't recover from bad retrieval — and the loop that can
Corrective RAG (CRAG) — grading the documents and correcting with web search
Self-RAG — reflection tokens that grade the model's own output
The honest, steep cost — and why you must route to this sparingly

From Pipeline to Loop

The core shift is one sentence: agentic RAG is not a pipeline; it is a loop. Where standard RAG flows strictly forward (retrieve → generate → answer), agentic RAG inserts decision points and can loop back:

retrieve → 🧪 grade docs → ✍️ generate → 🔬 grade answer → ✔️ done — but if the docs are bad, it loops back to retrieve (with a new query or a web search); if the answer isn't grounded, it loops back to regenerate.

This requires a cyclic graph, which is exactly why LangGraph became the standard tool for it in 2026. A LangChain chain is a directed acyclic graph — data only flows forward. LangGraph is a stateful, cyclic graph: nodes (retrieve, grade, generate, verify) each do one job, and conditional edges route execution — including backward — based on the results and shared state.

An infographic titled 'Agentic RAG — RAG as a Self-Correcting Loop'. A strip explains that standard RAG is a fixed pipeline (retrieve then generate) — if retrieval returns garbage it generates from garbage — whereas agentic RAG is a LOOP: the agent grades what it retrieved, corrects it, generates, then checks its own answer, retrying until confident. THE GRADE-AND-CORRECT LOOP, built as a cyclic graph with LangGraph, is a flow: Retrieve, then Grade docs, then Generate, then Grade answer, then Answer. Two correction branches: if docs are bad, rewrite the query or web-search and loop back to retrieve (CRAG); if the answer is not grounded, regenerate or re-retrieve (Self-RAG). Two method cards. Corrective RAG (CRAG): a lightweight retrieval evaluator grades the documents into Correct (refine and keep), Ambiguous (both), or Incorrect (discard then web search), fixing bad retrieval before generating. Self-RAG: the LLM emits reflection tokens to grade its own work — Retrieve (when to retrieve), ISREL (relevant?), ISSUP (grounded?), ISUSE (useful?) — retrieving on demand and catching hallucinations. A caveat: the price is steep — about 5 to 7 LLM calls per cycle (10 to 15 for multi-hop), about 3 to 4 seconds latency versus about half a second, and 4 to 8 times the cost; cap iterations so loops can't run forever, and route only hard or high-stakes queries here while single-hop lookups stay on standard RAG. Banner: stop generating from bad context — grade it, correct it, and check your own answer in a loop; robust, but reserve it for when it's worth the cost.

Read the graph and you'll see the whole lesson: the two grade nodes are the new intelligence, and the two conditional edges (loop back on a bad grade) are what make it self-correcting:

# Agentic RAG as a cyclic graph (LangGraph). Each node has one job; EDGES branch on results.
from langgraph.graph import StateGraph, END

g = StateGraph(RAGState)
g.add_node("retrieve",  retrieve)        # vector / hybrid search
g.add_node("grade_docs", grade_docs)     # CRAG: are these docs relevant?  (the retrieval evaluator)
g.add_node("websearch", web_search)      # CRAG correction: fetch better knowledge
g.add_node("generate",  generate)
g.add_node("grade_answer", grade_answer) # Self-RAG: grounded in the docs? does it answer the question?

g.add_edge("retrieve", "grade_docs")
g.add_conditional_edges("grade_docs",     # ← the loop: branch on the document grade
    lambda s: "generate" if s.docs_relevant else "websearch")
g.add_edge("websearch", "generate")
g.add_conditional_edges("grade_answer",   # ← and again on the answer grade
    lambda s: END if s.grounded and s.answers else "generate",   # not grounded? regenerate
)
g.add_edge("generate", "grade_answer")
# A linear chain (LangChain) can't do this — only a CYCLIC graph can route back to an earlier node.

Corrective RAG (CRAG): Grade the Documents

The first big idea — Corrective RAG (Yan et al., 2024) — attacks the retrieval failure: what if the documents you retrieved are irrelevant? Standard RAG never asks. CRAG adds a lightweight retrieval evaluator (a small fine-tuned model, e.g. T5-large) that grades the retrieved documents for a query and returns a confidence in one of three buckets — each triggering a different corrective action:

# CORRECTIVE RAG (CRAG): a lightweight evaluator grades the DOCS, then corrects.
def grade_docs(state):
    grade = retrieval_evaluator(state.question, state.docs)   # e.g. a fine-tuned T5 → confidence
    if grade == "Correct":
        state.docs = refine(state.docs)          # decompose-then-recompose: keep key strips, drop noise
        state.docs_relevant = True
    elif grade == "Incorrect":
        state.docs_relevant = False              # → the graph routes to web search
    else:  # "Ambiguous"
        state.docs = refine(state.docs) + web_search(state.question)   # combine both
        state.docs_relevant = True
    return state

Correct (docs are good): refine them — a decompose-then-recompose step strips the irrelevant sentences and keeps only the key knowledge, so even good documents get cleaned before generation.
Incorrect (docs are bad): discard them and fall back to web search. This is CRAG's signature move — a static corpus can only return what it has, so when it has nothing relevant, go get fresh knowledge from the web instead of hallucinating from junk.
Ambiguous (unsure): hedge — combine the refined internal docs and web results.

The effect: CRAG actively detects and fixes bad retrieval before generating, which directly cuts the hallucinations that come from feeding a model off-topic context. It's plug-and-play — a grade-and-correct layer you can wrap around any existing RAG retriever.

Self-RAG: Grade Your Own Answer

The second big idea — Self-RAG (Asai et al., 2023, "Learning to Retrieve, Generate, and Critique through Self-Reflection") — attacks a different failure: is my generated answer actually grounded — or did I just make it up? It trains a single LLM to emit special reflection tokens that let it control retrieval and critique itself mid-generation:

# SELF-RAG: the model emits REFLECTION TOKENS to control retrieval and critique itself.
#   [Retrieve]  → decide ON DEMAND whether to retrieve (not every query needs it)
#   [ISREL]     → is the retrieved passage RELEVANT?
#   [ISSUP]     → is my statement SUPPORTED by the passage?  (grounding / hallucination check)
#   [ISUSE]     → is the response USEFUL?
def grade_answer(state):
    supported = llm_judge(f"Is every claim in this answer supported by the context?\n"
                          f"Answer: {state.answer}\nContext: {state.docs}")   # ISSUP
    answers   = llm_judge(f"Does this answer the question '{state.question}'?")  # ISUSE
    state.grounded, state.answers = supported, answers
    return state   # if not grounded → the graph loops back to regenerate, strictly grounded

The four reflection tokens, each a learned self-check:

[Retrieve] — decide, on demand, whether retrieval is even needed ("not every query needs it" — chitchat and known facts don't). This alone cuts latency and noise.
[ISREL] (IsRelevant) — is each retrieved passage actually relevant?
[ISSUP] (IsSupported) — is the generated statement supported by the passage? This is the grounding / hallucination check — the model verifies its own claims against the evidence.
[ISUSE] (IsUseful) — is the response actually useful?

CRAG and Self-RAG are complementary, not competing. CRAG grades the inputs (the documents) and corrects retrieval; Self-RAG grades the outputs (the answer's relevance, grounding, usefulness) and corrects generation. A strong agentic system does both — the loop in the widget below grades documents and the final answer. Together they make RAG robust: bad retrieval gets corrected, and ungrounded answers get caught and regenerated.

See It: The Self-Correcting Loop

Watch the agent think. Pick a scenario and follow its trace — where it grades the documents (CRAG), corrects with a web search if they're bad, generates, then grades its own answer (Self-RAG) and regenerates if it isn't grounded. Keep an eye on the LLM-call counter and what standard RAG would have done instead:

Pick a scenario and watch the agent's trace: grade the docs (CRAG → web-search if bad), generate, then grade its own answer (Self-RAG → regenerate if ungrounded). Note the LLM-call count vs standard RAG — and what standard RAG would have done.

Notice the two failure modes the loop rescues: bad retrieval (CRAG routes to web search instead of hallucinating from off-topic docs) and a hallucinated answer (Self-RAG's grounding check catches it and forces a regenerate). In both, standard RAG would have confidently returned a wrong answer. That robustness is the entire value proposition — and the call-counter is the entire catch.

The Honest Take: Power Has a Price

Agentic RAG is the most powerful — and by far the most expensive — end of the RAG spectrum. Every grade, correction, and self-check is another LLM call, and they add up fast. Be brutally clear-eyed about the cost:

LLM calls explode. Standard RAG ≈ 1–2 model calls per query. A single-retrieval agentic cycle runs ~5–7 (route, reformulate, grade docs, generate, grade answer); a multi-hop query with two retrieval rounds hits ~10–15.
Latency multiplies. ~0.5s standard → ~3–4s agentic — a real UX hit for interactive chat.
Cost multiplies ~4–8×. Real teams report jumps like $200/mo →$ 1,800 in two weeks, or $1k →$ 10k+/mo at scale.
Loops can run forever. A self-correcting graph must have a hard iteration cap / budget (max retries) or it can cycle endlessly — the same hop-limit discipline as L101.
The graders can be wrong. If the document evaluator or the grounding judge mis-grades, you correct in the wrong direction — and you're paying extra to do it.

And the punchline from the benchmarks: on simple queries, agentic RAG often barely beats standard RAG while costing far more. One FAQ-bot study: standard RAG 94% correct, agentic 96% — but 8× the cost and 3s vs 0.4s. Not worth it.

So: don't make everything agentic. The production pattern is the one you already know — route (L101): send single-hop factual lookups to standard RAG, and escalate to the agentic loop only for multi-step reasoning, cross-source synthesis, or high-stakes queries (legal, medical, financial) where a wrong answer is costly and worth the latency. Cache intermediate steps, cap iterations, and reserve the self-correcting machinery for where it earns its keep.

🧪 Try It Yourself

Drive the loop, then make the call:

In the widget, run the bad-retrieval scenario. What corrective action does CRAG take, and what would standard RAG have returned instead?
Run the hallucinated-answer scenario. Which mechanism catches it — CRAG or Self-RAG — and which reflection token is doing the work?
On the happy path, agentic RAG and standard RAG return the same good answer. What did the agentic version cost to reach it, and what does that imply?
Your loop sometimes never returns an answer. What single safeguard is missing?
Decide standard vs agentic for each: (a) "what's our refund window?" (b) "draft a legal memo comparing how three contracts handle IP, citing clauses" (c) a 50k-QPS autocomplete; (d) "which of our suppliers are affected by the new tariff, and what's our exposure?"

→ (1) CRAG grades the docs Incorrect → discards them and web-searches for better knowledge; standard RAG would have generated from the irrelevant docs → a confident wrong answer. (2) Self-RAG — its [ISSUP] (IsSupported) grounding check finds the answer isn't supported by the docs and triggers a regenerate. (3) It cost ~3 LLM calls and several seconds for an answer standard RAG got in one call — i.e. on easy queries the agentic overhead is pure waste, which is why you route. (4) A hard iteration cap / budget (max retries) so the loop can't cycle forever. (5a) Standard (single-hop lookup). (5b) Agentic (multi-step, cross-document, high-stakes, auditable). (5c) Standard (sub-second, huge volume — agentic is economically impossible). (5d) Agentic (multi-hop synthesis, high-stakes).

Mental-Model Corrections

"RAG is retrieve-then-generate." That's standard RAG. Agentic RAG is a loop — retrieve, grade, correct, generate, self-check, retry — built as a cyclic graph (LangGraph), not a linear chain.
"If retrieval is bad, nothing can be done." CRAG grades the docs and corrects — refine them, or discard and web-search for better knowledge.
"The model can't know if it hallucinated." Self-RAG makes it grade its own answer ([ISSUP] grounding check) and regenerate if unsupported.
"CRAG and Self-RAG are alternatives." Complementary: CRAG fixes the inputs (documents), Self-RAG fixes the outputs (answer). Strong systems do both.
"Agentic RAG is strictly better — use it everywhere." It's 4–8× the cost and ~6× the latency, and barely helps on easy queries. Route to it only for hard / high-stakes / multi-hop.
"Just let the loop run until it's confident." Without a hard iteration cap, it can loop forever. Always bound it.

Key Takeaways

Agentic RAG turns RAG from a pipeline into a self-correcting loop — retrieve, grade, correct, generate, self-check, retry — built as a cyclic graph (LangGraph), which a linear chain can't express.
Corrective RAG (CRAG) grades the documents with a lightweight evaluator → Correct (refine), Ambiguous (combine), Incorrect (discard → web search). It fixes bad retrieval before generating.
Self-RAG emits reflection tokens — [Retrieve] (when to retrieve), [ISREL] (relevant?), [ISSUP] (grounded?), [ISUSE] (useful?) — so the model critiques its own work and catches hallucinations. CRAG fixes inputs; Self-RAG fixes outputs; use both.
The cost is steep: ~5–7 LLM calls/cycle (10–15 multi-hop), ~3–4s latency, ~4–8× cost. Always cap iterations, and route only hard / high-stakes / multi-hop queries here — single-hop lookups stay on standard RAG.
This is the bridge from RAG to agents — retrieval as iterative, tool-using reasoning.
Next: Evaluating RAG (RAGAS) — the faithfulness / relevance / context metrics that tell you whether any of these techniques actually improved your system.