Few-Shot Done Right (Choosing Examples)

Few-shot prompting — showing the model a handful of examples instead of describing what you want — is the single most powerful technique in the prompt-engineering toolkit. You met it in the In-Context Learning lesson. But here's the catch nobody mentions: its power depends entirely on which examples you pick.

Think of your examples as training data in miniature. The model imitates them — faithfully. Pick three near-identical easy cases and you teach it to handle only easy cases; pick three biased examples and you bake in the bias. Choosing examples well is a real skill, and it's the difference between few-shot that transforms output and few-shot that quietly misleads.

You'll learn the rules for choosing examples:

• Diversity over repetition, and covering edge cases
• Balancing labels and matching the real distribution
• Consistency of format, and the surprising power of order
• How many to use — and when to switch to dynamic selection

The mental model that makes everything click: a few-shot example is a tiny, on-the-fly demonstration the model copies. The model doesn't analyze your examples academically — it pattern-matches them and continues the pattern.

That means every property of your examples gets imitated: their format, their style, their difficulty, their label distribution, even their mistakes. So 'choosing good examples' isn't about finding nice examples — it's about curating a tiny dataset that demonstrates exactly the behavior you want across the range of inputs you'll actually see. The rules below all flow from this one idea.

Six rules cover almost everything. Keep this as your checklist whenever you write a few-shot prompt:

Three nearly-identical examples are a waste — they teach the format once and burn tokens repeating it. Three deliberately different examples teach the model the range it must handle: a clean/normal case, an awkward/messy case, and a genuine edge case.

• ❌ Three tidy product reviews, all clearly positive.
• ✅ One clearly positive, one mixed/ambiguous, one sarcastic-negative — so the model learns the whole spectrum, not just the easy end.

And this is your best tool for edge cases: if the model keeps mishandling a specific tricky input (an empty field, a foreign-language entry, a trick question), show it one example of exactly that case handled correctly. A single well-placed edge example often fixes a failure that paragraphs of instructions couldn't (recall: 'add the thing that fixes the failure you see').

The model picks up on the distribution of your examples, not just each one. Two traps:

• Label imbalance. For a classification task, if 4 of your 5 examples are labeled 'spam,' the model learns a bias toward 'spam.' Balance the labels so no answer is over-represented.
• Unrepresentative inputs. If your examples are all short, clean, easy inputs but production data is long and messy, the model is trained for the wrong world. Match your examples to the real inputs — same length, messiness, and variety you'll actually face.

Rule of thumb: your example set should be a fair, miniature sample of reality, not a cherry-picked highlight reel.

Your examples define a template, and the model copies it precisely — including any sloppiness. So format every example identically: the same field names, the same delimiters, the same structure, the same tone.

Input: "The food was cold."      → Sentiment: negative
Input: "Loved every bite!"        → Sentiment: positive
Input: "It was fine, I guess."    → Sentiment: neutral

If one example used Sentiment: and another Label -, or one wrapped input in quotes and another didn't, the model mixes the patterns and produces unpredictable output. Even subtle inconsistencies (capitalization, spacing, tone) leak through. Consistency in the examples buys you consistency in the answers — pair this with the delimiters/XML from last lesson for maximum reliability.

Here's a genuinely surprising one: the order of your examples can change the result dramatically. In classic experiments, reordering the same set of examples swung accuracy from near state-of-the-art to near-random — same examples, different sequence.

The driver is recency bias — the model weights examples nearer the end more heavily (the same 'edges matter' effect from the long-context and prompt-anatomy lessons). Two practical moves:

• Put your most representative / most important example last so it leaves the strongest impression.
• For classification, don't end on a run of the same label (e.g. three 'positive' in a row at the end) — it nudges the model toward that label. Mix the order.

If results feel unstable, try shuffling the example order before assuming the examples themselves are wrong.

More examples is not better. The sweet spot is 2–5.

• Too few (0–1) often isn't enough to convey a non-obvious pattern.
• 2–5 is where few-shot shines — enough to show the pattern and its range.
• Too many (8+) wastes tokens (cost + latency + context), can cause the model to overfit to your examples' quirks, and adds noise.

The workflow: start with 2–3, and add an example only to fix a specific failure you observe — not preemptively. And the key escalation signal: if you find yourself past ~8 examples and still struggling, few-shot is the wrong tool — you've graduated to either fine-tuning (bake the behavior into the weights) or dynamic example selection (next).

A static, hand-picked example set is perfect for prototyping. But in production, the best examples for a given request depend on what that request is — a billing question and a bug report want different demonstrations.

Dynamic (retrieval-based) example selection solves this: keep a pool of many labeled examples, and for each incoming input, retrieve the few most similar ones (via embedding/semantic similarity) and drop those into the prompt. Every request gets the most relevant few-shot examples, automatically.

This is essentially 'few-shot via retrieval' — and it's your first glimpse of the machinery behind RAG (embeddings + similarity search), which gets its own container later. For now, the takeaway: when one fixed set of examples can't cover a diverse workload, select examples per-input instead of hard-coding them.

Fix it with one example. Your few-shot prompt has three clean, clearly-positive review examples — and the model keeps mislabeling sarcastic reviews as positive. What's the single highest-leverage change?

→ Add one sarcastic-but-negative example. A targeted edge-case demonstration fixes a stubborn failure that more clean examples (or more instructions) never would. Diversity over repetition — and put your most representative example last (recency).

• "More examples = better." No — 2–5 is the sweet spot; 8+ adds noise/cost and risks overfitting. Past ~8, fine-tune or select dynamically.
• "Any examples will do." No — examples are training data in miniature; the model copies their format, range, and bias.
• "Pick the cleanest, easiest examples." No — include messy and edge cases; match the real distribution, or you train for the wrong world.
• "Order doesn't matter." It can swing results dramatically — recency bias; put your best example last, don't end on a label run.
• "Inconsistent formatting is fine if the content's right." No — even small format/tone drift makes the model mix patterns.

• Few-shot examples are training data in miniature — the model imitates everything about them, so choosing well is the work.
• Diversity over repetition: cover normal, messy, and edge cases; a single targeted example is the best fix for a stubborn failure.
• Balance labels and match the real input distribution — a fair miniature sample, not a highlight reel.
• Be ruthlessly consistent in format, and mind order — recency bias means your most representative example should go last.
• Keep it to 2–5; past ~8, switch to fine-tuning or dynamic (retrieval-based) example selection.

Next — the section finale: prompts are rarely perfect on the first try, so we learn to iterate on them systematically — testing, comparing, and improving prompts like an engineer, not by guesswork.