Defensive UX: Designing for Uncertainty & Failure

You've made the answer fast (L224), streamed it smoothly (L225), and made it verifiable (L226). This lesson is about the other half of every AI product: what happens when it goes wrong. Because it will — and AI goes wrong in a way classic software doesn't.

The reframe: design for failure as the default, not the exception. A normal app mostly works or throws a clear error. An AI feature fails improvisationally and confidently — it hallucinates a plausible wrong answer with no error thrown, stalls under load, gets refused by a guardrail, or takes an irreversible action on a bad inference. Defensive UX is the craft of designing the experience so that when the model is uncertain, wrong, slow, refused, or down, the product degrades gracefully, stays recoverable, and never lets a wrong or irreversible action through silently.

In this lesson:

• Why AI fails differently — and why exceptions aren't enough to catch it
• Graceful error states — transparency, recovery paths, and never losing the user's work
• Graceful degradation — the fallback ladder, and what to retry (and not)
• Reversibility & the right guard — the severity × reversibility matrix, undo, and confirmation
• Real human-in-the-loop — why a prompt isn't a gate
• Designing against over-reliance — calibrated friction, not blind trust

Scope: this is the product/UX view of failure. The engineering of reliability — retries with backoff, circuit breakers, timeouts, idempotency — has its own lesson (Fallbacks, Retries & Circuit Breakers), and agent-side recovery (tool errors, replanning) lives in the Agents material. Here we design what the user experiences. It builds directly on guardrails (L222), consistency/degradation (L224), and confidence/abstention (L226).

Classic software has a comforting property: it mostly works, or it errors clearly. A failed database write throws; you catch it. AI breaks that contract in a way that makes failure harder to see:

• It fails silently and confidently. The worst AI failure isn't a 500 — it's a fluent, plausible, wrong answer with no exception thrown. You can't try/except your way to catching it.
• It fails improvisationally. The same input can succeed at noon and hallucinate at midnight; failure depends on the prompt, the data, load, and randomness. There's no clean reproducer.
• It fails in three places, not one. Google's PAIR framework names them: system errors (the model/infra genuinely broke), context errors (the system worked, but the user's mental model was misaligned — it answered the wrong question), and user errors (the input was off). Most teams design only for the first.

The consequence for design: you cannot rely on error handling alone. A lot of AI failure never raises an error — it just produces something wrong-but-confident. So defensive UX has to assume "this output might be wrong even though nothing crashed" and build the experience around verification, reversibility, and recovery — not just exception catching. And, as PAIR puts it, when you do surface a failure, be human, not machine, and design the failure to be safe and boring — never make a dangerous failure interesting or exploitable.

When something does fail visibly, the difference between a trusted product and a broken-feeling one is almost entirely UX. The rules:

• Transparency beats silence. The two worst options are a raw Error 503 and a spinner that spins forever. Users are remarkably forgiving when they understand what's happening: "We're experiencing high demand — your request is queued" buys far more patience than a cryptic code or dead air.
• Acknowledge → explain → offer a next step. Not "Something went wrong," but "I'm not confident about this — want me to try again with more detail?" Explain why it couldn't answer and give a path forward (PAIR's "not enough data to predict prices yet — try again in a month").
• Never lose the user's input or workflow. This is the cardinal sin. An error must preserve what they typed and where they were. Make the last action retryable and let them regenerate without re-typing.
• Offer 2–3 real recovery options: retry · edit the input · regenerate · fall back to a manual path · hand off to a human. Return control to the user — don't dead-end them.

Concretely, that means catching the failure, degrading instead of hanging, and packaging a recoverable state that holds the user's work:

The test: if the model vanished mid-request, would the user lose their work or hit a dead end? If yes, you have a defensive-UX bug — independent of any model quality.

The best failure is one the user barely notices because the product quietly stepped down a rung instead of collapsing. Microsoft's HAX guidelines call this out directly — "scope services or gracefully degrade when the system is uncertain" — and the pattern is a ladder of fallbacks, best to worst:

Two engineering facts that shape the UX:

• Degrade on uncertainty, don't double down. If the model's confidence is low or evidence is thin, scope down (a narrower answer, a clarifying question, an abstention — L226) rather than confidently guessing. Compute the uncertainty and act on it.
• Only retry transient errors. Retrying a 429/5xx/timeout (with backoff) is right; retrying a 400, 401/403, or a context-overflow just burns time and money on something that will never succeed — degrade or fix instead.

This is the UX face of two earlier ideas: design to your p95 and cap-and-degrade (L224's consistency beats speed), and the fail-open vs fail-closed choice from guardrails (L222). The ladder is how you keep a bad moment from becoming a broken product. (The engineering of the rungs — circuit breakers, backoff — is the reliability lesson.)

Here's the framework that decides how much you should trust the AI to just act. Before an AI (or an agent) does something, ask two questions and let them pick the guard:

• Severity — how bad is the worst case if it's wrong?
• Reversibility — can you undo it?

That's a 2×2, and each cell has a natural guard:

And the single most useful move in defensive UX falls out of it: make irreversible actions reversible, and the guard they need gets lighter. The same task — "reply to the customer" — needs a hard gate if the AI auto-sends it, but no gate at all if the AI drafts it for review. The reversibility lever:

• Draft, don't send. Soft-delete, don't hard-delete. Stage, don't deploy.
• Undo toast after the action (the classic "Deleted — Undo" with a few-second window).
• A short delay before anything slow-to-reverse actually commits.
• Type-to-confirm ("type DELETE") for the genuinely irreversible.

Turn the dial from "the AI did something scary" to "the AI did something I can undo." The Action-Guard Lab below lets you classify real actions and feel exactly where the line is.

Six AI actions. For each, pick the guard — and watch the severity × reversibility matrix fill in. The pair to watch is #2 vs #3 (draft vs auto-send): same task, opposite guard, and the only difference is reversibility:

The matrix isn't bureaucracy — it's how you decide where to spend friction. Gate the irreversible, high-stakes actions; let the reversible, low-stakes ones run; and whenever you can, redesign an action to be reversible so it needs a lighter guard. Which raises the question the next section answers: what makes a gate real?

When you decide an action needs approval first, how you implement the gate matters enormously — and the most common implementation is fake.

Putting "always ask the user before deleting anything" in the system prompt is not a guard. The model can ignore it, be jailbroken past it, or hallucinate that it already got approval. A safety gate you can talk your way past isn't a gate.

A real human-in-the-loop gate lives outside the model: the thing that actually executes the action refuses to run it until an out-of-band approval clears. The policy lives in versioned config the model never sees; the model can only propose the action and surface its uncertainty:

Make the approval itself a good experience, or reviewers rubber-stamp it (which is the same as no gate):

• Give the reviewer everything inline — the exact action + parameters, the agent's reasoning, and the context — so they don't have to go re-derive the decision.
• Timeout = denial. Never let an unanswered request auto-approve.
• Offer a third option: approve / deny / request clarification — not just a binary.
• Confidence-route it: auto-approve genuinely high-confidence low-risk actions, escalate low-confidence or many-uncertainty-factor ones — so humans spend attention where it matters.

The principle: keep the safety decision out of the probabilistic part of the system. The model proposes; deterministic, auditable infrastructure disposes.

Defensive UX isn't only about the AI failing — it's about the human failing to catch it. Automation bias / over-reliance — accepting AI output without critical evaluation — is a named risk in the NIST Generative AI Profile ("Human-AI Configuration"), and it's a design problem: a technically great model still produces systematic over-reliance if the interface invites blind trust.

The defensive antidotes:

• Don't fake certainty. A confident-looking UI on an uncertain system manufactures over-reliance. Surface calibrated confidence and abstain when thin (L226).
• Add friction where it counts. A small, deliberate speed bump on high-stakes actions (a confirm, a diff to review, a required edit) keeps the human in the loop and engaged.
• But don't over-friction the trivial. Confirmation dialogs on harmless actions train users to click through everything — which destroys the friction exactly when you need it. Spend friction like a budget: on the irreversible, high-severity actions from the matrix.
• Keep the human doing the thinking. Show the work to review (a diff, the sources), invite edits, and make the human an editor, not a rubber stamp.

The balance is the whole game: enough friction and visible uncertainty that people stay critical, not so much that they tune it out. Calibrated trust — neither blind reliance nor reflexive distrust — is the target.

One failure mode is special because you chose it: the refusal. When a guardrail blocks an output (L222) or the model abstains because it can't ground an answer (L226), that moment is part of the product — and a curt "I can't help with that" feels as broken as a crash.

Design the refusal like any other error state:

• Explain, briefly and without preaching — why it's declining, at the right altitude (don't lecture, don't over-explain a safety boundary in a way that teaches people to bypass it).
• Offer a path — a safe alternative, a narrower version of the request, a human, or "here's what I can do." A refusal should be a fork in the road, not a wall.
• Keep it consistent — the same class of request should refuse the same way; erratic guardrails read as broken and erode trust.

A good abstention or refusal is defensive UX working as intended — it's the system declining to confidently do the wrong thing. Make declining feel like care, not a dead end.

Work these through, using the Action-Guard Lab where it helps:

1. In the lab, #2 (draft a reply) and #3 (auto-send it) are the same task with opposite guards. What single property flips them, and what's the design takeaway?
2. Your AI feature shows a spinner, the model call times out after 30s, and the user's typed prompt disappears. Name the three defensive-UX failures here.
3. A teammate secures a delete action by adding "never delete without asking" to the system prompt. Why isn't that a real guard, and what is?
4. Your model is overloaded and returning 503s. What should the user experience, and how is that different from what they should experience on a 400?
5. Users are pasting your AI's answers into production without checking them. The model is fine — so what's the design fix?

→ (1) Reversibility. A draft is reversible (nothing leaves until a human sends), so it needs no gate; auto-send is irreversible, so it needs approval. Takeaway: make actions reversible (draft-not-send, soft-delete, undo) to downgrade the guard they need. (2) A forever/long spinner (bound the timeout + show progress), no graceful degradation (fall down the ladder instead of timing out into nothing), and losing the user's input (never discard what they typed — preserve + offer retry). (3) A prompt instruction is inside the probabilistic model — it can be ignored, jailbroken, or have approval hallucinated. A real gate lives outside the model: the executor refuses to run the action until an out-of-band approval clears (and timeout = denial). (4) A 503 is transient — quietly retry with backoff and, if it persists, show "high demand, queued / try again" and degrade (cached/simpler). A 400/auth/context-overflow is permanent — don't retry; surface a clear, specific message and a fix/edit path. (5) Over-reliance is a design problem: add calibrated friction on consequential use, show uncertainty and the sources to review (L226), make the human an editor (a diff to approve), and don't present an uncertain answer with a certain-looking UI.

• “Just handle the errors.” Most AI failure throws no error — it's a confident wrong answer. Design for "might be wrong even though nothing crashed": verification, reversibility, recovery.
• “A spinner is fine while it loads.” A forever-spinner is worse than a clear message. Bound timeouts, show progress, and degrade — transparency beats silence.
• “Errors can reset the form.” Never lose the user's input. Preserve their work and make the action retryable / regenerable without re-typing.
• “Retry on failure.” Only on transient errors (429/5xx/timeout). Retrying 400/401/context-overflow just wastes time and money.
• “Confirm every action to be safe.” Confirming trivial actions trains click-through. Spend friction on high-severity, irreversible actions; make the rest reversible instead.
• “The system prompt says ask before deleting, so we're safe.” A prompt is not a gate — the model can ignore or fake approval. Real gates live outside the model.
• “Our model is accurate, so we're done.” Accuracy doesn't prevent over-reliance — design friction + visible uncertainty + reversibility so humans stay critical and can recover.

• Design for failure as the default. AI fails improvisationally and confidently — often with no error thrown — so build the experience around verification, reversibility, and recovery, not just exception catching.
• Graceful error states: transparency beats silence (no Error 503, no forever-spinner) — acknowledge, explain, offer a next step; and never lose the user's input (retry / regenerate / edit / manual / human handoff).
• Graceful degradation: a fallback ladder (full AI → simplified → rule-based → human), scope down on uncertainty instead of doubling down, and only retry transient errors (429/5xx/timeout).
• Match the guard to the action (severity × reversibility): autonomous / log & notify / undo window / approve first — and the key lever, make irreversible actions reversible (draft-not-send, soft-delete, undo) to need a lighter guard.
• A prompt is not a gate. Real human-in-the-loop lives outside the model (the executor refuses until an out-of-band approval clears; timeout = denial).
• Design against over-reliance: calibrated friction on high-stakes actions (not trivial ones), visible uncertainty, and the human as editor — calibrated trust, not blind reliance.
• Next — L228: Feedback UI & the Data Flywheel — turning thumbs, edits, and corrections into the signal that makes the product (and the model) better over time.