Shared State & Race Conditions: Why Locks Exist

The last lesson ended with a small horror: two workers each add 1 to a shared counter a hundred times, and instead of 200 you get 197. Or 183. Or a different wrong number every single run. No typo, no bug in your logic — the code is obviously correct. And yet it produces garbage.

Welcome to the single most treacherous problem in all of concurrent programming: the race condition. It's what happens when two workers touch the same piece of data at the same time, and it is responsible for some of the most expensive, most embarrassing, hardest-to-find bugs ever shipped — double-charged credit cards, oversold inventory, bank balances that go negative, user accounts silently overwritten. And the reason it exists is almost insultingly simple, once you see it.

This lesson does three things. First, it shows you exactly how a line as innocent as count = count + 1 corrupts data when two workers run it at once. Second — and this is the part that should scare you a little — it explains why this bug is nearly invisible: it passes every test, works perfectly on your laptop, and only strikes in production, under load, at 3 a.m. Third, it shows you the family of tools built to defeat it — starting with the oldest and most famous, the lock — and why they work.

Scope: we build the idea in plain terms, on one machine. The same problem across many machines at once — two servers editing the same row — is the hardest problem in distributed systems, and we return to it, armed, much later in the course.

Here is the whole bug, and it hides inside a line every programmer writes on their first day:

count = count + 1

It looks like a single, instantaneous action. It is not. To the machine, it's three separate steps — a pattern called read-modify-write:

1. Read the current value of count out of memory (say, it's 5).
2. Modify it — add 1 — on a private scratchpad, getting 6.
3. Write the new value (6) back into memory.

For one worker, this is flawless: read 5, make 6, write 6. Done. The danger lives entirely in the gap between step 1 and step 3 — the moment when a worker has read a value but hasn't written back yet. It's holding a number that's already out of date, and it doesn't know it.

With a single worker, that gap never matters, because nobody else is around to change count while it's mid-update. The instant you add a second worker running the same line at the same time, that harmless little gap becomes a trapdoor.

Picture the classic image. A shared whiteboard has a tally on it: 5. Two people, Ana and Ben, each want to add 1, at the same moment. Watch what happens when their steps interleave:

	Ana	Ben	Whiteboard
1	reads 5		5
2		reads 5 (Ana hasn't written yet!)	5
3	writes 6		6
4		writes 6	6

The board says 6. But two people each added one — it should say 7. Ben read the value before Ana wrote hers, so he was working from a stale 5, and his write blindly clobbered Ana's. One increment simply vanished. This is a lost update, and it's the heart of every race condition.

Now scale it up. Two workers, each adding 1 to a shared counter one million times, expecting 2,000,000. Run it and you'll get something like 1,455,032 — and a different wrong number each time you run it. Every one of those hundreds of thousands of missing counts is a moment when two workers landed in that read-write gap together. The math isn't broken. The timing is.

If the race condition just failed reliably, it would be annoying but manageable — you'd catch it in testing and fix it. What makes it genuinely dangerous is that it is non-deterministic: whether it corrupts anything depends on the exact microsecond timing of how two workers interleave, and that timing changes with load, with the machine, with the phase of the moon.

The practical consequences are brutal:

• It passes all your tests. In a test, one request runs at a time; the workers never collide, so the bug never fires. Green checkmarks all the way down.
• It works on your machine. Low load, few cores, gentle timing — it might run correctly a million times in dev.
• It destroys data in production. Under real concurrent load, workers finally collide, and the counter drifts, the balance goes wrong, the item oversells — intermittently, unreproducibly.

It even has a name that captures the cruelest twist: a Heisenbug. Try to observe it — add a log line, attach a debugger, slow things down to watch — and you change the timing just enough that the collision stops happening and the bug vanishes. Like the physics principle it's named for, the act of measuring it alters it. "But it works on my machine" is the anthem of the engineer who has just met a race condition and doesn't know it yet.

This is why you cannot fix races by testing harder. You have to fix them by design — by making the collision structurally impossible.

Reading about a lost update is one thing; watching a real one disappear in front of you is another. Below, two workers share a counter. Set how many times each should increment it (say, 1,000 each — so the correct total is 2,000), and press Run with the lock off.

Watch their read-modify-write steps interleave in real time, and watch the red flashes — each one is a lost update, two workers reading the same value. The final total lands below 2,000, and a different wrong number every time you run it. Now flip the lock on and run again: a single key passes between the workers, only the one holding it can touch the counter, the other visibly waits — and the total comes out to exactly 2,000, every single run. That difference, felt, is the whole lesson.

The cure follows directly from the diagnosis. The bug is two workers in that read-write gap together — so the fix is to make sure only one worker is ever in the gap at a time.

First, a name for the dangerous stretch: the critical section. It's the little run of code that reads shared data and then writes it back. Correctness breaks only when two workers are inside the same critical section simultaneously. So the entire goal is: let one worker through the critical section at a time, and make everyone else wait their turn.

The classic tool for that is a lock (also called a mutex, for "mutual exclusion"). Go back to the whiteboard, but now there's a single marker, and the rule is: you may only read and write the board while you're holding the marker. Ana grabs the marker, reads 5, writes 6, and hands it back. Only then can Ben grab it, read the now-correct 6, and write 7. The board is always right, because the read and the write can no longer be split apart by someone else — the lock makes the whole read-modify-write atomic, meaning indivisible.

In code it's just as small:

lock.acquire()        # take the marker (others wait here)
count = count + 1     # critical section — now safe
lock.release()        # hand the marker back

One key, one writer at a time. The collision that caused the whole problem is now structurally impossible — not "unlikely," impossible. (And if you're wondering how the lock itself avoids a race when two workers grab for it at once: the hardware provides one truly-indivisible instruction to build locks on. It's turtles all the way down to a single atomic step.)

A counter is the teaching example, but the same bug wearing a suit costs companies real money every day. The pattern to recognize is check-then-act: read a value, make a decision based on it, then act — with a gap in the middle.

The canonical business version is overselling inventory. One concert ticket left. Two fans hit "Buy" in the same millisecond:

Fan A: reads stock = 1 → "1 ≥ 1? yes" → writes stock = 0
Fan B: reads stock = 1 → "1 ≥ 1? yes" → writes stock = -1   ← sold twice

The ticket sold twice; someone shows up to a seat that doesn't exist. Same shape everywhere: two ATMs on one account both approving a withdrawal, a coupon redeemed twice, a "username taken?" check that lets two people grab the same name. Every one is a check-then-act race.

And a lock is only the most famous fix, not the only one. They're all just different ways to close the gap between read and write:

• An atomic operation does the whole read-modify-write as one indivisible step. A database can run UPDATE items SET stock = stock - 1 WHERE stock >= 1 and sell each unit exactly once — no app-side lock needed, because the database guarantees that statement can't be interrupted.
• A pessimistic lock — the database's SELECT ... FOR UPDATE — locks the row so no one else can read-to-modify it until you're done. A lock, one level down.
• An optimistic lock takes the opposite bet: don't lock at all. Read a version number with the data, and when you write, only succeed if the version hasn't changed. If someone beat you to it, your write fails and you retry. No waiting, no deadlocks — great when collisions are rare.

One idea underneath all of them: never leave a gap between reading and writing that another worker can slip into. Close it by exclusion (a lock), by indivisibility (an atomic op), or by detect-and-retry (optimistic).

Locks fix corruption, but they are not free — they trade one hard problem for a few softer ones, and knowing them is what separates "I added a lock" from "I added the right lock."

Locks serialize — and serialization is the enemy of speed. A lock's whole job is to force one-at-a-time. So every worker that wants the same lock has to queue up and wait, and right there, your beautifully parallel program becomes serial. That's not a metaphor — it's literally Amdahl's Law from the last lesson: the locked section is a serial fraction, and if it's big or hotly contested, it caps your speedup no matter how many cores you have. The discipline: make the critical section as small as possible — lock for the single line that touches shared state, not the whole function.

Locks can deadlock. Give workers two locks and you can get a standoff: worker A holds lock 1 and waits for lock 2; worker B holds lock 2 and waits for lock 1. Neither will ever let go. Both wait forever, and your system silently freezes. (The textbook picture is the dining philosophers, each grabbing one fork and starving for the second.) The classic defense is beautifully simple: always acquire locks in the same order everywhere in your code, so a cycle can never form.

Which leads to the highest-level lesson of this whole section: the safest shared state is no shared state. If two workers never touch the same data — give each its own copy and combine the results at the end — there's no critical section, no lock, no deadlock, no race. A huge amount of modern system design is exactly this move: avoid sharing mutable state so you never have to coordinate it. When you can't avoid it, reach for the cheapest correct fix — an atomic operation before a fine lock, a fine lock before a coarse one.

Here's the thing that makes this lesson quietly enormous: the race condition is not a single-machine problem. It's a shape that reappears at every scale, and once you can see it, you see it constantly.

Two threads updating one variable is the smallest version. Two processes on one server hitting one file is the same shape. Two application servers updating the same database row is the same shape. Two data centers on opposite sides of the planet trying to agree on one number — that's the same shape too, and it's so hard that solving it well earned Turing-level respect. Every one is "multiple actors, one piece of shared state, and the outcome depends on timing."

And the fixes rhyme all the way up. The in-memory lock becomes a database transaction, which becomes a distributed lock, which becomes a consensus protocol like Raft. The optimistic version number becomes the backbone of how distributed databases stay consistent. You've just learned the atom; the molecules and the chemistry come later in the course. But the atom never changes: wherever two things share mutable state, you must coordinate their access — or accept corruption.

"It passed all my tests, so it's safe." Race conditions are Heisenbugs — they pass ~always and fail rarely, only under real concurrent load. Testing almost never catches them; you prevent them by design.

"Races only happen with multiple cores." No. A single core time-slicing between tasks (concurrency, from the last lesson) can interleave a read-modify-write and lose an update just fine. Concurrency is enough to cause a race; parallelism just makes it more likely.

"Just wrap everything in one big lock." That makes it correct but slow — you've serialized your whole program (Amdahl) and invited deadlock. Lock the smallest critical section, consistently.

"Reads don't need protecting." A read that feeds a later write — check-then-act — absolutely does; that read-to-write gap is the race. (Purely independent reads often don't.)

"count += 1 is one operation." It's read-modify-write: three steps. That gap between read and write is the entire bug.

"Locks are slow, so avoid them." Correctness beats speed every time: an unprotected race silently corrupts data, which is far worse than a little slower. Use the cheapest correct fix — and best of all, avoid sharing mutable state so you don't need one.

Ten lines and a laptop turn this from a claim into a scar you'll never forget. Predict first: two threads each add 1 to a shared counter 1,000,000 times. What's the final value?

import threading

count = 0
def bump():
    global count
    for _ in range(1_000_000):
        count = count + 1        # read-modify-write — the gap is here

a = threading.Thread(target=bump); b = threading.Thread(target=bump)
a.start(); b.start(); a.join(); b.join()
print(count)                     # expected 2,000,000 …  got ??? (and different each run)

Run it a few times: you'll see numbers like 1,373,912 then 1,206,548 then 1,558,003 — never 2,000,000, and never the same twice. Now wrap the increment in a lock and watch it snap to exactly 2,000,000 every run:

lock = threading.Lock()
def bump():
    global count
    for _ in range(1_000_000):
        with lock:               # take the marker, add, hand it back
            count = count + 1

The gap between your first prediction and that random wrong number is the exact size of the danger shared state poses — and the lock is what closes it.

The most dangerous bug in concurrency, in five lines:

• A race condition is two workers touching shared data at once, where the result depends on timing. Its root is that read-modify-write (count = count + 1) is three steps, not one — and a second worker can slip into the gap and cause a lost update.
• It's a Heisenbug: non-deterministic, passes every test, works in dev, corrupts production under load, and vanishes when you try to observe it. You fix it by design, not by testing.
• The fix is to make the critical section run one worker at a time — with a lock, an atomic operation, or an optimistic version check. The one idea: close the gap between read and write.
• The bill: locks serialize (Amdahl again) and can deadlock — so keep critical sections tiny, order your locks, and best of all, don't share mutable state when you can avoid it.
• The shape scales all the way up — threads, processes, servers, data centers — and so do the fixes, from a mutex to a consensus protocol.

You've now spent a whole section deep inside a single server — its CPU, memory, disk, the way it holds connections, juggles work, and protects shared state. There's one question left that ties it all together: when a server finally does fall over under load, what exactly ran out? Every server dies one of a small number of deaths, and knowing which one is coming is the difference between a scaling plan and a prayer. Next, the finale of this section: The Four Bottlenecks — CPU, Memory, Disk, Network.