Showing Your Work: Citations, Sources & Confidence

You've made the answer fast (L224) and made it stream in beautifully (L225). Now make it trustworthy — because a fluent, well-streamed answer that the user can't verify is, in any serious setting, a liability.

The cautionary tale that opens this lesson: in Mata v. Avianca (2023), two lawyers filed a federal brief full of case citations that ChatGPT had fabricated — real-looking names, citations, and quoted passages for cases that did not exist. When challenged, they asked ChatGPT "are these real?" and it said yes. The judge sanctioned them $5,000 and dismissed the matter; it became the case that reset how every court thinks about AI. The failure wasn't just the hallucination — it was that nothing in the workflow let anyone verify the sources before it mattered.

This lesson is about showing your work: the product/UX craft of surfacing sources, citations, and confidence so users can verify what the AI says and calibrate how much to trust it. In this lesson:

• Why show your work — verifiability as the foundation of (appropriate, not blind) trust
• Citation UI patterns — inline numbers, source cards, hover-to-reveal-the-span, deep links
• Grounded attribution — where trustworthy citations come from (machine-checkable cited_text spans)
• The faithfulness gap — the hard truth that “cited” is not “verified”
• Showing reasoning — chain-of-thought displays, and their big caveat
• Confidence & abstention — calibrated uncertainty, and the power of a good “I don't know”

Scope: this is the product/UX view of trust. The retrieval-side mechanics of grounding an answer in documents — chunking, retrieval, the grounding prompt — were the RAG section's Citing Sources & Grounding Answers lesson; here we assume you can retrieve, and focus on how to present it so a human (or an automated check) can verify it. This is also the UX counterpart to the hallucination guardrail (L222).

LLMs are fluent by construction — they produce confident, well-formed prose whether or not it's true. That decouples how convincing an answer looks from whether it's correct, which is exactly the condition under which users get burned. Showing your work re-couples them by making the answer checkable. Concretely, citations and sources buy you four things:

• Verifiability — the user (or a downstream system) can follow a claim to its source and confirm it. This is the difference between a toy and a tool someone will stake a decision on.
• Appropriate trust, not blind trust — the goal isn't to make users trust the AI more; it's to help them trust it correctly — lean in when it's grounded, double-check when it's thin.
• Fewer hallucinations, real and perceived — models grounded in retrieved sources and asked to cite them hallucinate less, and even when they don't, visible sources let users catch the misses instead of absorbing them.
• Accountability & recourse — when something is wrong, a citation trail tells you where it went wrong (bad source vs bad synthesis) and gives the user somewhere to go.

The stakes scale with the domain. A cited summary of a blog post is a nicety; an uncited medical, legal, or financial claim is a hazard. The higher the cost of being wrong, the more “show your work” shifts from polish to requirement — and the more you should be willing to spend latency and screen space on it.

There's a spectrum of ways to surface sources, and they are not equally trustworthy. Roughly from least to most verifiable:

The jump in value is from “here's a source” to “here's the exact sentence that supports this claim.” That's the difference between a citation you have to go audit and one you can verify at a glance. Design notes that matter:

• Cite the sentence, not the document. Fine-grained, claim-level attribution is far more checkable than a bag of links at the bottom.
• Make it ambient, not noisy — inline superscripts that expand on hover/click keep the prose readable while the evidence is one interaction away.
• Stream citations with the text — as the answer streams in (L225), attach each citation to its claim as it lands, so the sourced answer assembles live.

Target the hover-the-span / deep-link end of this spectrum. A bare [4] looks authoritative but makes the user do all the work; showing the supporting span does the verifying for them — which, as the next sections show, is the only thing that actually catches the bad citations.

If you just ask a model “cite your sources” in the prompt, it will happily emit [1], [2] — and make them up (that's literally the Avianca failure). Trustworthy citations are grounded: tied to real spans of real documents you put in front of the model. The reference implementation is a grounded-citation API — e.g. Anthropic's Citations API — which returns machine-checkable spans instead of trusting the model to quote:

• You enable it per source document and the API chunks the document (by sentence, by default) to define the citation granularity.
• The response is interleaved: a stream of text blocks where each claim block carries the exact cited_text span and a location — char_location (plain text, 0-indexed, exclusive end), page_location (PDFs, 1-indexed pages), or content_block_location (your own pre-chunked blocks).
• cited_text is the actual source span, returned for you (and it doesn't count toward output tokens) — so you can render the supporting line directly, and a check can compare claim ↔ span automatically.

Why this beats prompt-based “please quote the source”: the spans are guaranteed to point at real text in your documents, the model is measurably more likely to cite the most relevant quote, and cited_text is cheaper. A few integration realities worth knowing:

• Granularity is yours. Auto-sentence-chunking is the default; pass custom content blocks (e.g. one per RAG chunk) when you want to control exactly what a citation can point to.
• It streams — a citations_delta attaches each citation to the current text block as it arrives (so it composes with L225's streaming).
• It does not combine with strict structured outputs — grounded citations interleave citation blocks with text, which conflicts with a strict JSON schema (you pick one).

The retrieval that finds the right chunk is the RAG section's job; this is the layer that carries the exact supporting span all the way to the UI so the user can see it. Which matters enormously, because of the next section.

Here is the most important — and most overlooked — idea in this lesson. A citation being present does not mean it supports the claim. This is not a rare edge case:

Across studies of long-form, cited AI answers, human annotators find that roughly 50–90% of citations do not fully support the claim they're attached to. Let that land: the majority of citations in long answers are, to some degree, unfaithful.

The research term is correctness vs faithfulness: an answer can be correct and still be unfaithfully cited, and — worse for trust — an answer can be wrong while wearing a perfectly real-looking citation. A citation can fail on three independent axes:

• Accessibility — does the source exist and resolve? (Avianca failed here: the cases were fabricated.)
• Topical alignment — is the cited source even about this claim? (A claim cited to a billing doc that never mentions it.)
• Factual support — does the source actually say what the claim says? (The answer says 99.99%; the cited source says 99.9% — a real source, a plausible number, a wrong claim.)

This is why a bare [4] is dangerous: a citation raises perceived trust whether or not it's earned, so an unfaithful citation manufactures false confidence — the user trusts it more precisely because it's cited. The only robust antidote is to show the supporting span (so a glance can check it) and, increasingly, to evaluate citation faithfulness automatically (retrieve the cited span and check that it entails the claim — the job of tools like RAGAS, attribution evaluators, and citation-validators).

Put bluntly: the goal of “show your work” is not to display citations — it's to display verifiable ones. A citation the user can't check is decoration. The interactive below lets you feel exactly how this slips past you.

Five claims, five real-looking citations. You decide which ones the source actually supports — then reveal the span and the verdict. Watch how easily a confident, on-topic, cited claim sails past:

Two of the five citations don't hold up — a hallucinated 99.99% (cited to a source that says 99.9%) and a topical-mismatch to a billing snippet — and the worst offender was flagged HIGH confidence. Notice what made them catchable: seeing the cited span next to the claim. With only the bare [n], every claim looked identical and equally trustworthy. That's the faithfulness gap in your hands — and the case for showing the span, not the number.

“Show your work” also covers the model's reasoning — the chain-of-thought / extended thinking that led to the answer. Surfacing it (a collapsible “Thinking” panel, or an agent's visible steps and tool calls) builds trust and is genuinely useful for math, multi-step logic, agents, and debugging. UX guidance:

• Default to collapsed in low-stakes chat (the answer is what they want); expand by default for high-stakes, educational, or agentic tasks where the path matters.
• Make the steps legible — show retrieval, tool calls, and intermediate decisions as a timeline, not a wall of tokens. (This doubles as the progress UI from L224 and previews agent UX in L230.)

But there is a critical caveat you must internalize and design around:

The reasoning a model shows is not a faithful transcript of how it actually computed the answer. It's a post-hoc, readable summary — and research repeatedly shows models can produce fluent, plausible reasoning that has little to do with what truly drove the output. A model can reach the right answer for reasons it doesn't state, or state reasoning it didn't use.

So treat shown reasoning as an explanation aid, not ground truth or a guarantee. Don't let a convincing “thinking” trace substitute for actual verification (the citations above are the verifiable part; the reasoning is the narrative). And never expose raw chains that may contain system-prompt secrets, PII, or internal tooling — show a cleaned, user-facing version.

The third pillar: tell the user how much to trust this particular answer. Models are fluent at every confidence level, so the UI has to carry the uncertainty signal. The research is clear and a little counterintuitive:

• Calibrated hedging beats both extremes. Confident-and-wrong destroys trust; reflexive “I can't be sure” on everything is useless. The win is hedging that tracks actual uncertainty — “I'm fairly sure X; I'm not certain about Y” — so the hedge itself is informative.
• Show uncertainty without overload. A confidence badge/bar/color or a one-line hedge beats dumping raw log-probs on the user (cognitive overload erodes trust as surely as hiding it does).
• Beware the calibration-collapse trap. A change that raises average accuracy can destroy the hedging signal — the system starts rating confident-wrong answers above hedged-right ones, and users lose the cue they relied on. Track calibration, not just accuracy.
• Confidence is not correctness (the Inspector's #4: HIGH confidence, flatly unsupported). A model's stated confidence is often miscalibrated/overconfident — treat it as a soft signal, and prefer grounded evidence (a real cited span) over a self-reported number.

The failure mode to avoid: a certain-looking UI on top of an uncertain system. It teaches users blind trust, which produces both over-reliance on bad answers and abandonment after the first public miss. If the system is unsure, the interface should say so.

The most trustworthy thing an AI product can sometimes do is decline. A calibrated “I don't know” — or, in a grounded system, “that isn't in the sources” — is a feature, not a failure:

• A safe refusal beats a fluent, unsupported answer, especially in legal/medical/financial settings where a confident wrong answer is worse than no answer.
• Ground, then abstain when the evidence is insufficient. The pattern (e.g. SURE-RAG): judge whether the retrieved evidence supports / refutes / is insufficient for the question, and abstain when support is absent — rather than papering over the gap with plausible text.
• Make the refusal a good experience. Don't dead-end the user: say what you don't know, what you do, and offer a next step (“I couldn't find this in your docs — want me to search the web / file a question?”). A helpful abstention preserves trust; a curt one feels broken.

This closes the loop with confidence and citations: if you can't cite it and you're not sure, say so. Abstention is what keeps the “show your work” promise honest when there's no good work to show.

Reason through these, using the Citation Inspector where it helps:

1. In the Inspector, which claim was the hardest to catch, and why — what made a wrong claim look trustworthy?
2. A teammate ships answers with a footer list of source links and says “we have citations now.” Why is that weak, and what one change makes it genuinely verifiable?
3. Your eval shows the model's answers are 90% correct but a reviewer says the citations are bad. How can both be true at once?
4. Marketing wants a green “98% confident” badge on every answer to boost trust. Give two reasons that can backfire.
5. A medical-info assistant gets a question its sources don't cover. What's the right behavior, and why is it better product design than a confident best-guess?

→ (1) Claim #4 (the 99.99% vs 99.9% SLA): it had a real, on-topic source, a plausible number, and HIGH model confidence — everything looked right except the one digit the source didn't support. Only seeing the span catches it. (2) A footer list is claim-agnostic — you can't tell which source backs which claim, so you can't actually verify any of them. Fix: inline, claim-level citations that reveal the supporting span (hover/click), not a pile of links. (3) Correctness ≠ faithfulness — the model can be right and cite sources that don't actually support the specific claims (right answer, wrong/unfaithful citations). Both metrics are real and you must measure them separately. (4) A blanket confidence badge is (a) miscalibrated — models are often overconfident, so “98%” won't track real accuracy — and (b) trust-destroying when wrong: a certain-looking UI on an uncertain system teaches blind trust, causing over-reliance and abandonment after the first confident miss. Show calibrated uncertainty, not a vanity number. (5) Abstain — “I couldn't find this in the approved sources” (plus a next step). In high-stakes domains a safe refusal beats a fluent unsupported answer; a confident wrong medical answer is the worst possible outcome.

• “It has citations, so it's trustworthy.” “Cited” is not “verified” — 50–90% of cited claims in long answers aren't fully supported. Show the span so it can be checked.
• “Just ask the model to cite its sources.” Prompt-only citations get fabricated (the Avianca failure). Use grounded citations tied to real document spans (cited_text + location).
• “A list of source links at the bottom counts.” That's claim-agnostic and barely verifiable. Do inline, claim-level attribution that reveals the supporting sentence.
• “Correct answer ⇒ good citations.” No — correctness ≠ faithfulness. Measure citation support separately from answer correctness.
• “Show the chain-of-thought and you've shown the truth.” Displayed reasoning is a post-hoc summary, not a faithful transcript — an explanation aid, not a guarantee (and may leak secrets/PII).
• “More confidence shown = more trust.” Only if it's calibrated. A certain UI on an uncertain system breeds blind trust → over-reliance and abandonment. Confidence ≠ correctness.
• “Always produce an answer.” Abstention is a feature — a grounded “not in the sources” beats a fluent unsupported answer, especially in high-stakes domains.

• Trust is a product feature. A fluent answer the user can't verify is a liability (Mata v. Avianca: $5k sanction for fabricated, fake-cited cases). Show your work so users can verify and calibrate trust.
• Citation UI is a spectrum — push from a footer link-list toward inline, claim-level citations that reveal the exact supporting span (hover/click) and deep-link to the source. Cite the sentence, not the doc.
• Ground your citations. Don't trust the model to quote — use grounded-citation APIs that return machine-checkable cited_text spans with char/page/block locations (it streams, and cited_text is free on output tokens).
• “Cited” is not “verified” (the key idea): 50–90% of cited claims in long answers aren't fully supported; correctness ≠ faithfulness. A bare [n] manufactures false confidence — show the span, and evaluate citation faithfulness.
• Showing reasoning builds trust but is a post-hoc summary, not a faithful transcript — an explanation aid, never a substitute for verifiable citations (and never leak secrets/PII).
• Confidence must be calibrated (it ≠ correctness; models are often overconfident); show uncertainty without overload, and abstain when the evidence isn't there — a safe “I don't know” beats a fluent unsupported answer.
• Next — L227: Defensive UX — designing for when the AI is wrong, slow, or unavailable: graceful errors, safe defaults, undo, and guardrail states that keep a broken moment from becoming a broken product.