Pessimistic vs Optimistic Locking

Isolation Levels & Their Anomalies ended on a loose thread: the cure for write skew (and for a dozen other races) is to "materialize the conflict" — take an explicit lock so two transactions actually collide instead of silently passing through. This lesson is about how you do that, and it turns out there are exactly two philosophies, splitting on a single question.

When two transactions both want to change the same row, you can bet one of two ways:

• Lock first — assume they'll conflict. Grab an exclusive lock before you touch the row; anyone else who wants it waits in line behind you. This is pessimistic locking: guilty until proven innocent.
• Check last — assume they won't conflict. Take no lock; do your work; and only at the moment you write, check whether anyone changed the row while you weren't looking. If someone did, throw your work away and retry. This is optimistic locking: innocent until proven guilty.

That's the whole taxonomy. Everything else — SELECT … FOR UPDATE, version columns, deadlocks, HTTP 412s, retry loops — is a detail of one bet or the other. And here's the punchline you should hold onto from the first sentence: neither is universally better. Which one wins is decided by how often the conflict you were betting on actually happens — the contention. Bet wrong and the price is steep and, as we'll measure, asymmetric. Let's take each bet precisely, then watch them race.

Pessimistic locking makes the conflict impossible by serializing access: before you read-to-modify a row, you take a lock on it, and you hold that lock until you commit. The workhorse in SQL is SELECT … FOR UPDATE, which takes an exclusive row lock (there are also shared locks for readers that must exclude writers). Anyone else who wants that row blocks until you're done. We measured exactly this on Postgres — one transaction held a FOR UPDATE lock for two seconds, and a second transaction's UPDATE on the same row took 2008 ms to return. It didn't fail; it waited. That wait is the whole cost model of pessimistic locking: safe by default, but every held lock is a potential queue. (The formal discipline that makes lock-based concurrency serializable is two-phase locking — acquire locks in a growing phase, release them only in a shrinking phase; strict two-phase locking, what real engines use, holds every lock to commit.)

Blocking is the tame failure. The signature hazard of pessimistic locking is the deadlock: two transactions each holding a lock the other needs, each waiting forever. It is not an exotic edge case — it is a structural consequence of letting transactions take multiple locks in different orders. We built one on purpose (T1 locks row A then wants B; T2 locks B then wants A) and Postgres caught it:

ERROR:  deadlock detected
DETAIL:  Process 138 waits for ShareLock on transaction 754; blocked by process 137.
Process 137 waits for ShareLock on transaction 755; blocked by process 138.

The database's deadlock detector found the cycle and aborted one transaction (the victim) to break it — the survivor commits, the victim must retry. You cannot wish deadlocks away; you manage them: acquire locks in a consistent global order (always A before B) so a cycle can't form, keep transactions short so locks are held briefly, and set a lock_timeout so a stuck transaction fails fast instead of hanging.

One more thing pessimistic locking can do, which surprises people: use locks to avoid contention. SELECT … FOR UPDATE SKIP LOCKED takes a lock but, instead of waiting on a row someone else already holds, skips it and grabs the next free one. We ran two workers against a job table and they cleanly took different rows — worker 1 got job 1, worker 2 got job 2, no blocking. It's the standard way to build a concurrent job queue on a plain database: many workers, one table, zero contention.

Optimistic locking makes a different bet: conflicts are rare, so don't pay for a lock you probably don't need. The name is a small lie worth clearing up immediately — optimistic locking does not take a lock at all. It's a version check at write time. The pattern (formalized by Kung & Robinson's 1981 On Optimistic Methods for Concurrency Control as three phases — read in a private workspace, validate at commit, write only if valid) is almost always implemented with a version column:

1. Read the row, and remember its version (say, 1).
2. Do your work — for as long as you like, holding nothing.
3. Write with the version as a guard: UPDATE … SET …, version = version + 1 WHERE id = ? AND version = 1.

If nobody touched the row, the version is still 1, the UPDATE matches, and you win. If someone committed first, the version has moved to 2, your WHERE matches nothing, and the write affects zero rows. That zero is the conflict signal. We watched it on Postgres — the first writer got UPDATE 1, and a second writer still holding the stale version got UPDATE 0:

-- both read version = 1, then:
UPDATE counter SET val = val+1, version = version+1
  WHERE id = 1 AND version = 1;   -->  UPDATE 1   (winner; version is now 2)
UPDATE counter SET val = val+1, version = version+1
  WHERE id = 1 AND version = 1;   -->  UPDATE 0   (CONFLICT — retry)

No lock was ever held; no one ever blocked. The price shows up only when the bet is wrong: on a conflict you must reload the row and redo your work. That's the retry, and building it is your job, not the database's — a loop with exponential backoff and a maximum retry budget so a hot row can't spin your service forever. The upside is real: no lock overhead, no blocking, and — because you never hold two locks at once — no deadlocks, ever. The whole cost is wasted work when you lose the race, which is cheap when races are rare... and ruinous when they're not. Which is exactly the trade the next section measures.

Now put them on the same track. We ran both strategies through pgbench against a single hot row on an 8-core Postgres, with real work in each transaction, and turned up the number of concurrent workers. The result is the whole lesson in one chart:

workers	pessimistic (tps)	optimistic (tps)	winner
1	478	499	optimistic — no lock cost
2	478	461	pessimistic
8	455	379	pessimistic
16	427	307	pessimistic (+40%)
64	332	299	pessimistic

At one worker — no contention — optimistic wins (499 vs 478): with nothing to conflict with, its lock-free path is pure upside, no lock to take, nothing to retry. But watch what happens as contention climbs. The crossover comes almost immediately, and then the gap widens: by 16 workers, pessimistic is ~40% faster (427 vs 307). Optimistic's throughput collapsed from 499 to ~300; pessimistic's only sagged from 478 to 427.

This is the part that matters, and it's not symmetric. When contention is high on one hot row, optimistic thrashes: most transactions read, do their work, try to commit, discover the version moved, and throw all that work away — we measured ~50% of attempts aborting and retrying, burning CPU to accomplish nothing. Pessimistic just makes them queue, so each unit of work is done exactly once. Optimistic's failure mode is a retry storm that can drive useful throughput toward zero (a livelock); pessimistic's failure mode is a line that grows but keeps moving. One degrades catastrophically; the other degrades gracefully.

There's a crucial nuance the same benchmark revealed, so you don't over-learn the rule: the crossover only bites when a wasted retry throws away real work. When we made each transaction a trivial bare counter++, optimistic stayed ahead even at a 50% abort rate — because redoing a sub-millisecond increment costs almost nothing. So it's not "high contention → pessimistic" flatly; it's "high contention and non-trivial work per attempt → pessimistic." Cheap conflicts are survivable; expensive ones are not.

Once you see "lock first vs check last," you find it far outside the database — because it's a general answer to any shared-resource race, the same problem Shared State & Race Conditions: Why Locks Exist introduced. The optimistic bet especially shows up anywhere a lock would be absurd:

• Version columns are the raw form, and frameworks bake them in: Hibernate/JPA's @Version annotation makes one integer column a logical clock — it auto-checks it on write and throws OptimisticLockException on a mismatch, which you catch, reload, and retry.
• HTTP does optimistic concurrency over the network. A GET returns an ETag (a version of the resource). To update, you send If-Match: <etag> on your PUT/DELETE; if someone changed the resource since your GET, the ETag no longer matches and the server returns 412 Precondition Failed. That's a version column bet, wearing an HTTP header — it prevents two web clients from silently clobbering each other (the lost update of the previous lesson, at the API layer). You literally cannot hold a database lock across a user staring at a form for five minutes; optimistic is the only sane choice for human-speed edits.
• Every "someone else edited this, reload?" prompt — wikis, Jira, Google Docs, your CMS — is an optimistic conflict surfaced to a person.

And the pessimistic bet generalizes just as cleanly — any time you take a lock (or a lease, or a mutex) before acting because a collision is likely or a retry is unacceptable. The two bets are a lens you'll keep reusing: is a conflict here likely enough, and expensive enough to retry, that I should pay to prevent it — or rare enough that I should gamble and just check?

The crossover is the kind of thing you have to feel the shape of, not memorize, so here it is as a live dial. Both strategies pound the same hot row; you turn the contention up and watch the winner flip. See the pessimistic workers queue for the lock, one at a time, while the optimistic ones race in and bounce back as retries — the abort percentage climbing on screen — and watch the two throughput curves diverge on the chart until optimistic dives below.

Then flip work per attempt from real to cheap and watch the whole conclusion change: with a nearly-free retry, optimistic stays ahead even under brutal contention. That's the difference between memorizing "pessimistic for high contention" and actually understanding why — and it's the difference between a senior engineer who reaches for the right tool and one who cargo-cults a rule.

You don't pick one strategy for your whole system; you pick it per access pattern. The decision is refreshingly concrete:

• Reach for optimistic when conflicts are rare (users editing different rows — different documents, different accounts), when you can't hold a lock across the operation (a human editing a form, a long multi-step flow, an HTTP request/response gap), and for read-heavy work. It's the default for most CRUD.
• Reach for pessimistic when conflicts are frequent and on the same row (a hot inventory count, a shared counter, a popular seat), when a retry is expensive (a long or side-effecting transaction you really don't want to redo), or when you need guaranteed exclusive access. Keep the transaction short — read, compute, write, commit — because you're holding a lock the whole time.
• Measure, don't guess. The single best signal is the retry rate under real load: if fewer than ~5% of your writes ever retry, optimistic is almost certainly right; if retries are climbing into the double digits, contention has crossed over and it's time to lock — or to remove the hot row entirely (shard the counter, batch the writes).

The takeaways:

• Two bets on one question: lock first (pessimistic — assume conflict, others wait) or check last (optimistic — assume none, retry if wrong). Optimistic takes no lock; it's a version check (UPDATE … WHERE version = N → 0 rows = conflict, measured).
• Each has one signature hazard: pessimistic → deadlocks (measured deadlock detected; fix with lock ordering + short txns + timeouts) and blocking (a FOR UPDATE made a writer wait 2008 ms); optimistic → retry storms (the loop is your job — backoff + a budget).
• Contention picks the winner, asymmetrically. Measured: optimistic wins with no contention (499 vs 478 tps) but collapses under it (down to 307 at 16 workers, ~40% behind), because it thrashes redoing wasted work; pessimistic queues gracefully (478 → 427). And it only bites when the wasted work is expensive — cheap retries keep optimistic ahead.
• It's everywhere: version columns, JPA @Version, HTTP ETag/If-Match → 412, and SELECT … FOR UPDATE SKIP LOCKED for lock-based job queues.

So far, every lock has lived inside one database, where a deadlock detector and a single commit can save you. Next — The Distributed Transaction Problem: what happens when the two things you need to change atomically live on different machines, and there is no shared lock manager, no single commit, and no one to detect the deadlock?