Self-Consistency & Majority Voting

Last lesson, chain-of-thought let the model reason before answering. But it has a weakness we flagged: a single chain of thought can take a wrong turn — one slip early on, and the whole answer is wrong, even on a problem the model could solve.

Self-consistency fixes this the way a careful person double-checks hard work: solve it several different ways, and go with the answer you keep landing on. Concretely, you sample many reasoning chains for the same question and take the majority vote of their answers. It's one of the most reliable accuracy boosts for hard reasoning — if you can afford it, which is the part most tutorials skip.

You'll learn:

• Why one chain isn't enough
• The sample-many-then-vote recipe (temperature, N)
• Why majority voting works (errors scatter, truth converges)
• The honest cost / diminishing-returns trade-off (a 2026 reality)
• Universal Self-Consistency for free-form answers, and when to use any of this

Recall CoT's limitation: once the model starts a reasoning chain, that trajectory is fixed — it follows the path it began and has no built-in way to notice 'wait, I made an arithmetic slip three steps ago.'

Because generation is probabilistic (the sampling lessons), running the same hard problem twice can produce different chains — most landing on the right answer, but some wandering into a wrong one. With a single sample, you're gambling that this particular run didn't take a wrong turn. On a genuinely hard, multi-step problem, that's a real risk.

The insight that fixes it: don't bet on one run — take a vote across many.

The technique is beautifully simple:

1. Prompt with chain-of-thought, but set temperature > 0 (so each run reasons a little differently).
2. Sample the same question N times → get N independent reasoning chains.
3. Extract the final answer from each chain.
4. Take the majority — the answer that appears most often wins.

The magic is in why this works: there are many valid routes to a correct answer, but countless idiosyncratic ways to be wrong. So the correct answer tends to converge (lots of chains reach it), while errors scatter (each wrong chain is wrong differently). Majority voting cancels the noise. Watch it on our discount problem:

In practice:

• Temperature: set it to ~0.6–1.0 so the chains actually differ. At temperature 0 every run is (nearly) identical, so voting tells you nothing — you need diversity for the vote to mean anything.
• Number of samples (N): more samples = more robust, but with sharply diminishing returns. 5–10 is usually plenty; research-grade results sometimes use 20–40, but you rarely need that.
• Aggregate: extract each final answer and take the mode (most frequent). For numeric/labels this is a simple tally.

That's the whole method — it's an ensemble for reasoning: many weak-ish independent guesses combine into a stronger one (the same 'wisdom of crowds' that powers ensembles in classic ML).

Here's the honest, up-to-date picture — and the reason self-consistency is a scalpel, not a default:

• It multiplies cost and latency. Sampling N chains means N× the tokens, N× the dollars, N× the wait (Iron Triangle, in full force). 10 samples = 10 separate generations.
• Gains plateau early. Accuracy improvements flatten after a handful of samples; pushing N higher mostly buys cost, not correctness.
• It can even hurt on easy problems. On questions the model already nails, extra samples just add noise and occasionally outvote the right answer.
• It's losing its edge (2026). On today's strong, often reasoning-capable models, the benefit is smaller than it was for GPT-3-era models — so don't apply it reflexively.

The takeaway: reserve self-consistency for genuinely hard, high-value problems where a wrong answer is costly and you can afford the compute — not as a blanket 'make it better' switch. Use a small N and consider early-stopping (stop once an answer clearly leads).

Plain majority voting only works when answers are discrete and extractable — a number, a label, 'yes/no.' You can't tally votes over five different essays or summaries.

Universal Self-Consistency (USC) extends the idea to free-form outputs: generate the N candidates as usual, then ask an LLM to pick the most consistent / representative answer among them, instead of exact-match voting. The model acts as the aggregator.

So the pattern generalizes: sample diversity, then converge — by voting when answers are discrete, or by LLM selection when they're open-ended. (Note this adds yet another model call on top of the N samples — more cost — so the same 'reserve for hard problems' rule applies.)

A quick decision guide:

• ✅ Use it for: hard math/logic/multi-step problems, high-stakes answers where reliability beats cost, and cases with a checkable, discrete answer — when you can spend N× compute to de-risk a wrong result.
• ❌ Skip it for: simple/lookup tasks, anything latency-sensitive (real-time chat), and high-volume/cost-sensitive paths — the cost rarely justifies the marginal gain.
• On reasoning models: it's largely internalized. Recall from the reasoning lesson that parallel thinking — sampling several solution paths and reconciling them — is essentially self-consistency built into test-time reasoning. So you often get its benefit automatically with a reasoning model, without orchestrating samples yourself.

Think of self-consistency as 'spend more inference compute to buy reliability' — powerful exactly when the problem is hard enough to deserve it.

Vote on a hard one. Take a problem your model gets wrong ~half the time. Sample it 5× at temperature≈0.7 and tally the final answers. Did the majority answer beat any single run?

Then the trade-off: that reliability cost you 5× the tokens & latency. When is that worth it? → Only for hard, high-value problems with a checkable answer — not as a default. (And on easy items it can even add noise.)

• "Use temperature 0 for the samples." No — you need diversity (temp ~0.6–1.0); identical chains make the vote meaningless.
• "More samples is always better." No — gains plateau early and can add noise; 5–10 is usually enough, and cost scales linearly.
• "It works on any output." Plain voting needs a discrete, extractable answer; for free-form, use USC (LLM picks the most consistent).
• "Apply it to every prompt to boost quality." No — it's a scalpel for hard, high-value problems; it's wasteful (and sometimes harmful) on easy ones.
• "It's a separate trick from reasoning models." Related — reasoning models' parallel thinking is essentially self-consistency built in.

• Self-consistency samples multiple CoT chains (temperature > 0) and takes the majority answer — robust because correct paths converge while errors scatter.
• Recipe: temperature ~0.6–1.0, N ≈ 5–10, extract each answer, take the mode (an ensemble for reasoning).
• The catch: N× cost & latency, diminishing returns (gains plateau early; can add noise on easy items) — so reserve it for hard, high-value, checkable problems, not as a default.
• For free-form outputs, use Universal Self-Consistency (an LLM picks the most consistent candidate).
• On reasoning models, this is largely built in via parallel thinking — you often get it for free.

Next: we move from reasoning to acting — ReAct, the pattern that interleaves reasoning with tool use, letting a model think and take actions (search, calculate, call APIs) to solve tasks.