Citing Sources & Grounding Answers

Introduction

In the last lesson your rag() already emitted a [1] and an honest "I don't know." This lesson makes that trustworthy — because a citation the user can't verify, or that points at the wrong place, is worse than no citation at all.

This is the heart of why RAG exists. A bare LLM says "trust me"; a grounded RAG system says "here's the answer, and here's the exact passage it came from — check for yourself." That shift — from confident assertion to verifiable claim — is the entire value proposition. Get it right and users trust your system; get it wrong and one confidently-wrong, well-cited answer destroys that trust.

In this lesson you'll learn:

What grounding really means, and how it differs from answer relevance
The citation ladder — four techniques from cheap-and-shaky to strong-and-verified
Native Citations APIs (the big 2026 upgrade) and structured citations in code
That citations can lie too — and how to verify them — plus how to measure grounding

Grounding vs. Relevance: Two Different Things

First, untangle two ideas beginners conflate — because an answer can ace one and fail the other:

Grounding (faithfulness): is every claim in the answer supported by the retrieved context? An answer is ungrounded if the model added facts from its training memory or invented them — even if those facts happen to be true. Grounding is about attribution, not correctness.
Answer relevance: does the answer actually address the question the user asked?

A model can be relevant but ungrounded ("Refunds take 3 days" — fluent, on-topic, but your docs say 5 and it guessed), or grounded but irrelevant (faithfully quotes a passage that doesn't answer the question). You want both, and you must track them separately (the RAGAS framework splits exactly these axes).

Grounding is the property; citations are the proof. Grounding makes the answer trustworthy; a citation makes that trust checkable. The rest of this lesson is about producing citations you can actually rely on.

An infographic titled 'Grounding & Citations — From Trust Me to Check for Yourself'. A definition strip says grounding means every claim is traceable to a retrieved source (not the model's memory, not invented), and citations are the mechanism that makes grounding verifiable. Below is the CITATION LADDER from weaker to stronger grounding, with reliability dots. Rung 1, Prompt and cite [n]: instruct the model to cite, but it can cite the wrong chunk or one that doesn't support the claim — cheapest, least reliable. Rung 2, Structured plus quote: return answer plus citations with id and quote, so the model must quote the supporting span and you can check it — parseable, verifiable quote. Rung 3, Native Citations API: Anthropic Citations, Vertex grounding, OpenAI annotations return span-level references guaranteed to point to your documents — reliable pointers, often cheaper. Rung 4, Post-hoc verify: check each claim against its source with an NLI or verifier LLM and refuse or retry if unsupported — strongest, extra call. Three cards below: Measure two different things — faithfulness (is every claim supported?) times answer relevance (does it address the question?), via RAGAS or an LLM judge; Citations can lie too — a model may cite a real chunk that doesn't support the claim, so verify the claim against the span; and Show your work — footnote markers, hover to preview the chunk, deep-link to the document and page. A banner reads: an answer without a verifiable source is a confident guess; grounding plus citations are why RAG beats a bare LLM.

The Citation Ladder: Four Techniques

There isn't one way to cite — there's a ladder from cheap-and-shaky to strong-and-verified. Climb it as far as your stakes demand.

Rung 1 — Prompt & cite [n] (reliability ◗◯◯◯). What you did last lesson: instruct "cite the chunk number you used." Cheap and zero-infra — but the model can cite the wrong chunk, or cite a real chunk that doesn't actually support the claim. Fine for low-stakes; never trust it blindly.

Rung 2 — Structured output + quote (◗◗◯◯). Don't just ask for a number — make the model return the exact supporting sentence in a structured shape. Now you can programmatically check the quote exists.

Rung 3 — Native Citations APIs (◗◗◗◯). Let the provider do the hard part (next section).

Rung 4 — Post-hoc verification (◗◗◗◗). After generating, check each claim against its cited source and refuse/retry if unsupported (two sections down).

Start with rung 2 for most apps — it's a small change with a big reliability jump:

from openai import OpenAI
from pydantic import BaseModel
client = OpenAI()

class Citation(BaseModel):
    chunk_id: int            # which numbered chunk the claim came from
    quote: str               # the EXACT sentence in that chunk that supports it

class GroundedAnswer(BaseModel):
    answer: str
    citations: list[Citation]
    answered: bool           # False ⇒ context didn't contain the answer

resp = client.chat.completions.parse(
    model="gpt-5.5",
    temperature=0,
    messages=[{"role": "user", "content": build_prompt(question, hits)}],
    response_format=GroundedAnswer,        # ← force the shape, no free-form parsing
)
ans = resp.choices[0].message.parsed

By forcing {answer, citations:[{chunk_id, quote}]} you get a machine-checkable result: you can confirm each quote really appears in its chunk before you ever show the answer. The answered: bool field also makes abstention explicit — "I don't know" becomes data, not a string you parse.

Native Citations APIs — The 2026 Upgrade

The biggest recent shift: the model providers now do citation extraction for you, at the span level, with guarantees. Anthropic's Citations API (GA on the Anthropic API and Google Cloud Vertex AI) lets Claude ground answers in the documents you supply and return references to the exact sentences and passages it used. Google offers grounding with citations on Vertex/Gemini; OpenAI returns annotations from file_search. The industry has converged here.

Why this matters: with a native API, the citation is guaranteed to be a valid pointer into the documents you provided — the provider can't return a span that isn't really there. That kills an entire class of bug (the pointer hallucination) and is often cheaper and more reliable than prompt-based citing. You still feed the relevant documents (that's your retrieval job); the API handles extracting and attaching the citations.

# Anthropic Citations API — the provider extracts span-level citations for you,
# guaranteed to point at real text in the documents you supplied.
import anthropic
client = anthropic.Anthropic()

msg = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{
        "role": "user",
        "content": [
            {"type": "document",
             "source": {"type": "text", "media_type": "text/plain", "data": doc_text},
             "title": "handbook.pdf",
             "citations": {"enabled": True}},          # ← turn citations on
            {"type": "text", "text": "How long do refunds take?"},
        ],
    }],
)
# Each response block carries `citations` with the exact cited_text + its location.
for block in msg.content:
    for cite in (block.citations or []):
        print(block.text, "→", repr(cite.cited_text))

The division of labor is the key idea: you retrieve the right documents (everything from the last lessons), and the provider guarantees the citations point at real spans within them. That's rung 3 — strong, low-effort grounding. (Caveat: a guaranteed-real span still needs to actually support the claim — which is rung 4.)

Trust, but Verify: Citations Can Lie

Here's the trap that catches teams who think "we added citations, we're done": a citation can be present, well-formatted, point at a real chunk — and still be wrong. Two failure shapes:

Wrong pointer: the model cites [2] but the fact actually came from [3] (or from its memory). Native APIs (rung 3) eliminate this — the span is guaranteed real.
Unsupporting source: the model cites a real chunk that doesn't actually support the claim. No API prevents this — only verification does.

So the strongest grounding verifies the claim ↔ span relationship after generation:

def verify(answer: GroundedAnswer, hits) -> bool:
    """Cheap guardrail: every cited quote must actually appear in its chunk."""
    by_id = {i + 1: h["text"] for i, h in enumerate(hits)}
    for c in answer.citations:
        source = by_id.get(c.chunk_id, "")
        if c.quote.strip() not in source:        # the model cited a span that isn't there
            return False                          # → refuse, retry, or flag for review
    return True

# Stronger (and slower): ask an NLI model / a verifier LLM
#   "Does this quote ENTAILS this claim?"  →  catches "real source, wrong claim".

The cheap version (above) checks the quote literally appears in its chunk — catches fabricated quotes instantly, no extra model call. The stronger version asks an NLI model or a verifier LLM "does this passage entail this claim?" — catching the subtle "real source, but it doesn't say that" case. When verification fails, you refuse, retry, or flag for human review rather than ship a confidently-wrong answer. Match the rung to the stakes: a hobby chatbot can live on rung 1–2; a medical or legal assistant belongs on rung 4.

Measure It & Show It

Measure grounding — you can't improve what you don't track. Two scores, matching the two axes:

Faithfulness = (claims supported by context) ÷ (total claims). The anti-hallucination score.
Answer relevance = does the answer address the question?

Tools like RAGAS (and LLM-as-judge) compute these over an eval set, so a prompt or retrieval change shows up as a number going up or down, not a vibe. (Add context precision/recall to measure the retrieval side — we'll go deep on evaluation later.)

Show your work in the UI — grounding the user can see. The best technique is wasted if it's invisible:

Footnote markers ([1]) next to each claim.
Hover-to-preview the exact source chunk — no click needed.
Deep links to the original document + page (this is why you captured page numbers at ingestion!).
Optional confidence / 'unverified' flags when verification is uncertain.

Visible, checkable sources are what turn a skeptical user into a trusting one — and notice the whole pipeline pays off here: the page number you grabbed at ingestion becomes the deep link at answer time.

🧪 Try It Yourself

Pick the right rung, then catch the bad citation:

Choose a technique for each system: (a) a weekend Discord bot answering FAQ from one wiki page; (b) a clinical assistant summarizing patient guidelines for a doctor; (c) a customer-support bot where answers must link to the exact policy page.
Judge these answers — is each grounded, relevant, both, or neither? Chunk [3] says "Refunds are processed within 5 business days."
- (a) "Refunds take about a week." [3]
- (b) "Our office is open 9–6 PT." [3]
- (c) "Refunds are processed within 5 business days." [3]

→ (1a) rung 1–2 (low stakes); (1b) rung 4 — verify every claim, refuse if unsupported (lives are involved); (1c) rung 3 (native API for guaranteed span links) or rung 2 with deep-linking. (2a) Relevant but not grounded — "about a week" ≠ the cited "5 business days"; the citation is a lie (unsupporting/contradicting). (2b) Grounded-looking citation but irrelevant — real chunk, wrong topic; doesn't answer refunds. (2c) Grounded and relevant — the only trustworthy one, because the claim matches its cited span. The lesson: a citation being present proves nothing — only claim ↔ span agreement does.

Interactive: a grounding-vs-relevance judge. A fixed retrieved source chunk [3] ('Refunds are processed within 5 business days.') anchors four candidate answers, and for each the user judges two independent axes — Grounded? (is every claim supported by the source) and Relevant? (does it address the question) — landing the answer in the 2x2. The cases cover every quadrant: a faithful cited answer (grounded + relevant), a fluent answer that cites [3] but says '30 days' (relevant but UNGROUNDED — the dangerous case, since the number came from the model's memory), a faithful quote answering the wrong question (grounded but irrelevant), and an off-topic invention (neither). Each judgment is scored with an explanation. The lesson's core distinction made tangible: grounding is about attribution, not correctness, so a cited, plausible answer can still be ungrounded — which is why you measure faithfulness and answer relevance separately and verify the claim against its span. Deterministic.

Mental-Model Corrections

"Grounding = correctness." No — grounding = the answer is supported by the retrieved context. A true fact pulled from the model's memory is still ungrounded. Track faithfulness and answer relevance separately.
"We added [n] citations, so we're trustworthy." Prompt-based citations (rung 1) can point at the wrong chunk or an unsupporting one. Climb the ladder; verify.
"A citation that points to a real source is correct." A real chunk can still not support the claim. Native APIs guarantee the pointer, not the entailment — that needs rung-4 verification.
"Citing is just a UI nicety." It's the mechanism that makes RAG verifiable — the whole reason it beats a bare LLM. And it's only useful if the user can see and check it.
"More citations = better." Better is accurate citations whose spans support the claims — and abstaining ("I don't know") when the context doesn't.

Key Takeaways

Grounding = every claim is traceable to a retrieved source; citations make grounding verifiable — "trust me" → "check for yourself." It's why RAG beats a bare LLM.
Separate faithfulness (claims supported?) from answer relevance (addresses the question?) — and measure both (RAGAS / LLM-judge).
Climb the citation ladder: 1) prompt & cite [n] → 2) structured {answer, citations:[{id, quote}]} → 3) native Citations APIs (Anthropic / Vertex / OpenAI — span-level, guaranteed pointers) → 4) post-hoc verification (claim ↔ span; refuse/retry). Match the rung to the stakes.
Citations can lie: a real chunk may not support the claim. Verify the quote exists (cheap) or that the span entails the claim (NLI / verifier LLM).
Show your work — footnotes, hover-preview, deep links to doc + page (the metadata you captured at ingestion pays off here).
Next: the diagnostic capstone — the common RAG failure modes, and how to trace each one back to its box.