Hallucination Mitigation Strategies

In L232 (Output Guardrails: Format, Safety & Grounding) we built the gate that catches an ungrounded claim on its way out the door. This lesson is the cure: the strategies that make the model produce fewer of them in the first place — and that know when to stop and admit it doesn't know.

Start with the uncomfortable truth, because it reframes everything: you cannot drive hallucination to zero. OpenAI's September 2025 paper Why Language Models Hallucinate shows it's not merely a bug to be patched — it's an incentive problem. Training and almost every benchmark score answers like a multiple-choice exam: a confident guess that's sometimes right beats an honest "I don't know" that scores zero. So models learn to guess fluently. You can't out-prompt a structural incentive.

Given that, the job isn't elimination — it's a layered playbook with three moves:

• Reduce — make a hallucination less likely (ground the model in facts; decode against the context).
• Detect — catch the ones that slip through (self-check, verify, measure uncertainty).
• Abstain — when the truth isn't available or confidence is low, say so.

And the deepest idea in the lesson, the one the lab is built around: match the fix to the cause. Why would the model make this up? If the fact is retrievable, ground it. If the model knows it but recall is shaky, sample for consistency. If the truth lives nowhere, no strategy can conjure it — only abstention is correct. (Where these failures come from is its own lesson, Where Hallucinations Come From; here we fix them.)

It's tempting to treat hallucination like a defect you'll eventually engineer away. The 2025 research says otherwise, and internalizing why is what makes you design the right system:

• Pre-training optimizes next-token likelihood over a corpus — there's no signal that says "and when you don't actually know, decline." The model learns the shape of confident text either way.
• Evaluation rewards guessing. Under the 0/1 accuracy scoring behind most leaderboards, a model that always guesses when unsure outscores one that abstains — because a guess is sometimes right and an abstention is always zero. We have, collectively, trained models to bluff.
• The fix is incentives, not just tricks. The long-term answer is to reward calibrated abstention — score uncertainty, penalize confident errors more than honest *"I don't know"*s.

The practical takeaway for you, the engineer: stop chasing a zero you can't reach, and instead engineer the system around an imperfect model — layer the reductions, add detection, and make abstention a first-class outcome your product handles gracefully (callback to L227 — Defensive UX: Designing for Uncertainty & Failure: an honest "I'm not sure" with a path forward beats a confident wrong answer every time).

Organize the toolbox by where in the lifecycle each technique acts — it's a ladder from cheapest-and-most-effective down to last-resort, and you climb as far as your risk budget requires:

(The research literature bins these into six families — training, architectural, prompt-optimization, post-generation QC, interpretability/uncertainty, and agent-orchestration — but the lifecycle view is what you reach for when building.) The rest of the lesson walks the three moves that matter most in production: reduce, detect, abstain.

Here's the most important sentence in the lesson: most hallucinations are missing knowledge. The model was asked something it doesn't reliably know, and — per the incentive above — it guessed. The fix is almost embarrassingly direct: give it the facts so it doesn't have to recall them.

• Retrieval-Augmented Generation (RAG) is the #1 mitigation, full stop. Retrieve the relevant source and put it in the context, then instruct the model to answer only from it. A grounded answer with the fact in front of it is dramatically less likely to be invented — and it can cite (which feeds the L232 grounding gate and L226 — Showing Your Work: Citations, Sources & Confidence). The whole RAG section of this course is, in a real sense, a hallucination-mitigation system.
• Grounded / context-aware decoding. Even with the source in context, a model can drift back to its parametric priors ("I'm pretty sure it's 90 days"). Decoding methods like Context-Aware Decoding (CAD) and DoLa bias generation toward the provided context (and the higher, more factual layers) so it trusts the document over its memory.
• Lower the temperature for factual tasks. High temperature = more sampling randomness = more room to wander off the facts. For creative tasks you want it; for factual ones, turn it down.

The mental shortcut: give the model the answer, don't make it remember the answer. Recall is where it hallucinates; reading is where it's reliable. But note the ceiling — RAG only helps if the fact is retrievable. If it isn't in any source you can reach, grounding has nothing to ground to (the lab's "Nowhere" scenario), and you'll need detection and abstention instead.

Some mitigation lives in how you prompt — cheap, no infrastructure, surprisingly effective:

• Chain-of-Verification (CoVe). Instead of trusting a first draft, the model fact-checks itself in steps: (1) draft an answer, (2) plan a set of verification questions about its own claims, (3) answer those questions independently (so the answers aren't biased by the draft), (4) revise the answer to fix any inconsistencies it found. It measurably reduces hallucination because the verification questions are easier to answer correctly than the original open-ended one.
• Cite-or-decline. Instruct the model to attach a source to each factual claim — and if it can't cite one, to say so rather than assert it. This turns "sounds right" into "shown right," and it's the prompt-side partner of the L232 grounding gate.
• Give it an out. Explicitly tell the model "if the context doesn't contain the answer, say 'I don't have that information.'" Models bluff partly because the prompt never gave them permission to abstain. Granting it is one line and it works.

These are reductions, not guarantees — a model can still verify wrongly or cite a real source that doesn't actually support the claim (the faithfulness gap from L226). So you pair them with detection.

Reduction lowers the odds; detection catches what still gets through, before it reaches the user. Two powerful, complementary ideas:

• Consistency-based self-checking. A model that genuinely knows an answer gives the same answer across stochastic samples; one that's confabulating gives different ones each time. So sample the answer N times and check agreement:
  • Self-consistency — sample multiple reasoning paths and take the majority; disagreement is a red flag.
  • SelfCheckGPT — sample several responses and score how consistent the claims are (via NLI / overlap / an LLM judge). It's black-box (no logits, no external KB needed), which makes it easy to bolt onto any API.
  • RARR — retrieve evidence for the draft's claims and revise to match it (detect and repair).
• Uncertainty estimation. Instead of N full generations, estimate the model's confidence and distrust low-confidence answers. The landmark method is semantic entropy (Farquhar et al., Nature 2024): cluster the model's samples by meaning (not exact wording) and compute entropy over those clusters — high semantic entropy = the model is unsure = likely confabulation. Because sampling N times is expensive (5–10×), Semantic-Entropy Probes approximate it from the hidden states of a single generation.

All of detection costs extra compute — more samples, more calls. That's the trade you tune in the lab's cost meter. The payoff: a confidence signal you can act on — which leads straight to the last move.

Every technique above can fail, and one situation defeats all of them: the truth simply isn't available — it's private, never written down, or doesn't exist. RAG retrieves nothing. CoVe's verification questions are themselves unanswerable. Self-consistency just makes the model consistently wrong. In that world there is exactly one correct output: "I don't know."

• Abstention is a feature, not a failure. A calibrated "I don't have a source for that" is the single highest-value behavior in a trustworthy AI product. It's the direct application of the 2025 incentives finding: you, the system designer, reward abstention where the benchmark didn't — via the prompt ("say IDK"), via a confidence threshold (abstain when semantic entropy is high), and ideally via fine-tuning.
• Make the threshold a product decision. How uncertain before you decline depends on stakes — a medical or legal answer should abstain readily; a brainstorming tool can afford to speculate (and label it). Same severity dial as the guardrail lessons.
• Design the abstention well. "I don't know" shouldn't be a dead end — offer a next step (escalate, search, ask a clarifying question). That's the defensive-UX craft from L227.

The whole lesson compresses to one rule: a confident wrong answer is the worst possible outcome — worse than a slow answer, worse than "I don't know," worse than an error message — because the user can't tell it's wrong. Mitigation exists to make sure that, when the model would otherwise bluff, the system either grounds it, catches it, or declines.

Now make the trade-offs yourself. One risky question; you decide where the truth lives, then stack mitigation strategies and watch the answer, the risk gauge, and the cost change live:

Run the three scenarios and the lesson lands on its own:

• In the docs → turn on RAG: risk collapses and the answer cites a source. Grounding is the biggest lever.
• In the model → add self-consistency / low temperature: you catch the unstable guess and stabilize the recall.
• Nowhere → stack everything except abstain: risk barely moves and the model fabricates confidently — no strategy can invent a missing fact. Add cite-or-abstain and it finally says "I don't know."

That contrast is the whole point: match the fix to the cause, and keep abstention as the floor.

The trap teams fall into is treating "reduce hallucination" as one knob. It isn't — the right move depends entirely on why the model would make this particular thing up:

Build the stack to fit your failure profile: a RAG product over a fixed knowledge base leans hard on grounding + the L232 faithfulness gate; an open-domain assistant leans on uncertainty + abstention. There's no one config — there's the config that matches your causes, with abstention underneath all of it as the safety net.

Use the Hallucination Mitigation Lab and what you've learned:

1. Set “In the docs” with no strategies, then turn on RAG. What happens to the risk and the answer, and why is RAG called the #1 lever?
2. Set “Nowhere” and stack RAG + Chain-of-Verification + self-consistency. Why does the risk barely move?
3. From there, add cite-or-abstain. What does the model now do, and why is that the correct outcome?
4. In your own words: why can't you reduce hallucination to zero, and how should that change what you build?
5. A confident wrong answer, a slow answer, and "I don't know" — rank them worst-to-best for a banking assistant, and say why.

→ (1) Risk collapses and the answer arrives with a citation — because most hallucinations are missing knowledge, and RAG hands the model the fact so it doesn't have to recall (and guess). (2) Because none of those strategies can invent a fact that isn't there — RAG retrieves nothing, CoVe's verification questions are unanswerable, and self-consistency just makes the model consistently wrong. They reduce recall errors, not missing-truth errors. (3) The model abstains — "I don't have a source for that" — which is the only correct output when the truth isn't available; a confident answer would be fabricated. (4) Because hallucination is an incentive problem (training + benchmarks reward confident guessing over honest uncertainty, OpenAI 2025), not a patchable bug. So you stop chasing zero and instead layer reduce + detect + abstain around an imperfect model. (5) Worst → best: confident wrong (the user acts on a falsehood they can't detect) → slow (annoying but correct) → "I don't know" (honest, and you can escalate). For banking, a confident hallucination is a real-money incident; abstention is a feature.

• “A good enough model won't hallucinate.” No — it's an incentive problem baked into training and benchmarks; you can't reach zero. Engineer the system, not just the model.
• “Hallucination mitigation is one setting.” It's a layered playbook — reduce, detect, abstain — and the right layer depends on the cause.
• “Just tell it not to hallucinate in the prompt.” A prompt can grant permission to abstain and add verification, but it can't override the model's lack of knowledge. Ground it or let it decline.
• “RAG fixes hallucination.” RAG fixes retrievable-fact hallucination — its biggest source. It does nothing when the truth isn't in any reachable source.
• “Self-consistency / verification catches everything.” They catch unstable guesses; a model that's confidently and consistently wrong sails through. Pair with grounding and uncertainty.
• “Abstaining means the product failed.” A calibrated "I don't know" is the highest-trust behavior — and far better than a confident falsehood the user can't detect.
• “This is the same as the output grounding gate.” That (L232) detects an ungrounded claim at the exit; this lesson reduces how often the model produces one and decides when to decline.

• You can't reach zero: hallucination is an incentive problem (training & benchmarks reward confident guessing — OpenAI, 2025). Engineer a layered playbook, not a silver bullet.
• Reduce — ground it: RAG is the #1 lever (most hallucinations are missing knowledge — give it the facts, don't make it recall); add grounded / low-temperature decoding (CAD, DoLa) so it trusts the context over its priors.
• Reduce — prompt it: Chain-of-Verification (draft → verify → revise), cite-or-decline, and explicitly grant permission to abstain.
• Detect — self-check & measure: real knowledge is stable across samples (self-consistency, SelfCheckGPT, RARR); semantic entropy turns that into a confidence signal — at the cost of extra compute.
• Abstain — the floor: when the truth isn't available or confidence is low, say "I don't know" — a confident wrong answer is the worst outcome. Reward calibrated abstention; design it as a feature (L227).
• Match the fix to the cause: retrievable → RAG; shaky recall → self-consistency; truth nowhere → abstain. This is the cure that complements the gate in L232 (Output Guardrails: Format, Safety & Grounding).