Concurrency vs Parallelism

We owe you a confession. For the last four lessons — threads, the event loop, connection pools — we've thrown around the words concurrent and parallel almost interchangeably. That looseness is fine for casual talk, but it hides a distinction that is behind a genuinely huge share of performance confusion: why adding threads sometimes helps and sometimes doesn't, why a single-threaded server like Node can feel fast, why eight cores rarely make anything eight times quicker.

Concurrency and parallelism are not the same thing. They're not even the same kind of thing — one is about how you structure a program, the other about how it executes — and mixing them up is how engineers end up adding workers to a problem that workers can't fix.

This is the lesson that names the idea sitting underneath everything in this section. Once you can tell the two apart cleanly, a pile of earlier facts snaps into focus: the event loop, the GIL, threads across cores, even why a small connection pool beat a big one. And you'll be able to answer the question that actually matters in a design review — "will more workers make this faster?" — with a confident yes or no instead of a shrug.

Scope: we build the distinction and the one question that decides which you need. The moment two of those parallel workers touch the same piece of memory, a new problem is born — corruption — and that's the very next lesson (Shared State & Race Conditions).

The cleanest definition comes from Rob Pike, and it's worth memorizing word for word:

Concurrency is about dealing with lots of things at once.
Parallelism is about doing lots of things at once.

Read it twice, because the difference between dealing with and doing is the whole lesson. Concurrency is about structure — how you carve a program into independent tasks that can make progress in overlapping stretches of time. Parallelism is about execution — those tasks actually running at the very same instant, on separate hardware.

Put concretely: concurrency is a property of your program; parallelism is a property of the machine it runs on. You write concurrency — you design the work as separable pieces. The hardware provides parallelism — if it has multiple cores, those pieces can literally run simultaneously. A concurrent design is what makes parallel execution possible, but the two can exist without each other: you can be concurrent without being parallel, and — rarely — parallel without being cleanly concurrent.

The two everyday images to hold onto for the rest of this lesson:

• One cook, four pans. A single cook juggling four dishes on one stove — stirring one, then flipping another, then checking the oven — is concurrent. The dishes overlap in time; the cook handles one thing at any given instant.
• Four cooks, four stoves. Four cooks each making one dish at the same moment is parallel. Four things are literally happening at once.

Here's the fact that surprises people: a single-core machine can be concurrent, but it can never be parallel. A single core runs exactly one instruction at any instant — one thing, truly one. So how does your laptop feel like it's running a browser, a music player, and a download all at once on a single core?

Time-slicing. The operating system's scheduler hands each task a tiny slice of the core — a few milliseconds — then rips it away and gives the next task a slice, then the next, cycling through them dozens or hundreds of times a second. Each task runs a little, pauses, runs a little more. Because the switching is so fast, it looks simultaneous to you — but at any single instant only one task is actually executing. The tasks overlap in time; they do not run at the same time. That's the difference between dealing with four things and doing four things.

This is the one cook at the stove. He isn't stirring four pans in the same instant — that's impossible with two hands. He's interleaving: a few seconds on this pan, a few on that one, moving fast enough that all four dishes progress and nothing burns. He is concurrent, not parallel — and for a kitchen where most of the time is spent waiting (for water to boil, for the oven), one attentive cook is astonishingly effective.

Every single-threaded system you've met is exactly this. The event loop from the 10,000-connections lesson? One cook, thousands of pans — pure concurrency, zero parallelism. It never does two things at once; it just never stands idle while one of them waits.

Parallelism needs more hardware. To actually do two things at the same instant, you need two things that can execute — two CPU cores, two machines, two of something. Give a task to core 1 and another to core 2, and now — for real, not as an illusion — two instructions execute in the same clock tick. That's parallelism: four cooks at four stoves, four dishes genuinely cooking at once.

And here's the beautiful part, the reason Rob Pike cares so much about the distinction: a good concurrent structure is automatically parallelizable. If you've already carved your program into independent tasks (concurrency), then handing those tasks to four cores instead of one requires... nothing new. The structure was the hard part; the parallel execution is free once the hardware shows up. That's why Pike says the goal of concurrency isn't speed — it's good structure. Speed is the bonus that good structure unlocks when you have cores to spare.

But — and this is the hinge the rest of the lesson turns on — parallelism only helps if the work can actually be done at the same time. Four cooks are four times faster at chopping four separate vegetables. They are not four times faster at boiling one pot of water, because you can't split "wait for the water" across four people. Whether more workers speed you up depends entirely on what kind of work you're doing — which is the single most useful question in this whole topic.

This is a thing you have to feel, so below you get to run the experiment. You have four tasks. Give each one a type — ⏳ waiting (an I/O task, mostly idle, like a network call) or ⚙ computing (a CPU task, mostly crunching) — then predict how long they'll take on one worker versus four, and hit Run.

Watch the one-worker lane interleave the tasks (a cursor hopping between them, time-slicing) while the four-worker lane runs them side by side. Then run the two experiments that make the whole lesson click: set all four tasks to ⏳ waiting — one worker nearly ties four, because the waits overlap for free. Now set all four to ⚙ computing — suddenly four workers finish in a quarter of the time, because only real parallelism speeds up real work. Then chain a dependency and watch even four workers crawl, because a chain can't be split.

Everything you just felt in the simulator collapses into one question you should ask of any slow system — the single most practical idea in this lesson:

Is the bottleneck waiting (I/O-bound) or computing (CPU-bound)?

I/O-bound work spends most of its time waiting — on the network, the disk, a database, another service. The CPU is barely working; it's twiddling its thumbs. For this, concurrency is enough, and usually the better tool. One worker with an event loop can interleave thousands of these tasks, because while one waits, the worker serves another — you're overlapping idle time, not competing for compute. Adding cores barely helps: four cooks don't boil water any faster.

CPU-bound work spends its time computing — resizing images, crunching numbers, hashing, parsing. The CPU is pinned at 100%. For this, concurrency alone does nothing — interleaving four compute-heavy tasks on one core just makes each take four times longer in turn. You need real parallelism: split the work across cores so they compute simultaneously. Four cores, four times the crunching.

This one distinction explains a mountain of real behavior. It's why Node.js and nginx — single-threaded event loops — are brilliant for typical web serving (I/O-bound: mostly waiting on databases and networks) and terrible for video encoding (CPU-bound). It's why Python's GIL (from the threads lesson) barely matters for a web API but cripples number-crunching. Match the model to the bottleneck: waiting wants concurrency, computing wants parallelism. Reach for the wrong one and you add complexity for zero speed.

Even when work is CPU-bound and genuinely parallelizable, there's a hard ceiling on how much faster more cores can make it — and it catches everyone off guard. Buy an 8-core machine expecting 8× and you'll get maybe 4–5×. The reason has a name: Amdahl's Law.

The insight is simple. Almost no job is 100% parallelizable. There's always some serial part — setup, reading the input, the final step that combines everyone's results, a section where step B needs step A's answer — that can only run one-at-a-time no matter how many cores you own. And that serial slice sets a ceiling. If a fraction f of the work is serial, the best speedup you can ever get, even with infinite cores, is 1 / f.

Make it concrete. A job that's 90% parallel, 10% serial:

• On 8 cores, the speedup is 1 / (0.1 + 0.9/8) ≈ 4.7× — not 8×.
• On infinite cores, it's 1 / 0.1 = 10×. That's the ceiling. Forever.

Ten percent serial caps you at 10×; a 5% serial part caps you at 20×; 1% at 100×. The serial fraction is destiny — and because that last serial bit (merging, coordinating, the one dependent step) refuses to shrink, piling on cores gives brutal diminishing returns. This is the deep reason behind a pattern you've now seen three times: more threads (lesson 4), a single-thread event loop being fine (lesson 5), a small pool beating a big one (lesson 6). Past a point, the un-parallelizable part dominates, and more workers just stand around.

Here's the payoff for having built up threads, event loops, and pools first: with this one distinction, they all become the same picture seen from different angles.

• The event loop (10K connections): one thread, thousands of connections. Pure concurrency — it deals with thousands of waiting sockets by interleaving, and never runs two in parallel. Perfect, because holding connections is I/O-bound.
• The GIL (processes vs threads): Python threads give you concurrency but not parallelism for CPU work — they take turns holding one lock. Great for I/O-bound waiting, useless for CPU-bound crunching (for which you reach for processes — real parallelism).
• Threads across cores: genuine parallelism — separate threads on separate cores, computing at the same instant. The right tool for CPU-bound work.
• The small connection pool (reuse beats re-open): sized to the database's real parallel capacity (its cores). More connections than the DB can run in parallel don't add throughput — Amdahl and coordination overhead eat them. Small-beats-big is this lesson.

One vocabulary, four lessons. Concurrency is the structure that lets a system deal with many things gracefully (and shines on I/O-bound waiting); parallelism is the hardware doing many things at once (and is the only thing that speeds up CPU-bound work, up to Amdahl's ceiling). Nearly every scaling decision in this course is, underneath, a choice between these two.

Given all that, the temptation is to reach for concurrency and parallelism everywhere. Resist it — because both charge a real price, and neither is free speed.

Parallelism bills you in coordination. The moment multiple workers cooperate on shared data, they have to synchronize — take turns, lock things, message each other — and every one of those coordination points burns time and, worse, invites bugs. Split work too finely and the overhead of handing out tasks and collecting results costs more than the work itself — your parallel version runs slower than the simple serial one. The dream case is embarrassingly parallel work — fully independent chunks that never talk (resize 10,000 images) — which scales beautifully. The nightmare is work full of dependencies and shared state, which barely scales at all.

Concurrency bills you in complexity. A concurrent program has more moving parts, more interleavings, more ways for two things to step on each other — which is exactly the failure mode of the next lesson.

So the discipline is the opposite of reflexive: don't add concurrency or parallelism until you have a bottleneck that demands it. If your latency and throughput already meet the requirement, adding either is premature optimization that trades real simplicity for imaginary speed — and hands you races and deadlocks in the bargain. Measure first, find whether the bottleneck is waiting or computing, and only then pick your tool.

"Concurrency and parallelism are the same thing." No — concurrency is dealing with many things (structure, interleaving); parallelism is doing many things (simultaneous execution). A single core can be concurrent; it can never be parallel.

"More cores (or threads) always means faster." Amdahl's Law caps it at 1/(serial fraction), and coordination overhead can make it slower. Only the independent, parallelizable part speeds up.

"Async makes my code parallel / run faster." Async gives you concurrency on one thread — it speeds up I/O-bound work by overlapping waits, and does nothing for CPU-bound work.

"Concurrency is a performance technique." Its real purpose is good structure — cleanly separable tasks. Speed (via parallelism) is a bonus the structure can unlock, not the goal.

"To go faster, just parallelize everything." Only CPU-bound, independent work benefits — and only within Amdahl's ceiling and coordination cost. I/O-bound work wants concurrency instead; dependent work can't be sped up at all.

"A single-core machine can't do many things at once." It can be concurrent — time-slicing creates the overlap. It just isn't parallel.

Ten minutes and a laptop settle the whole lesson. Predict first: for a job that just waits, will 4 parallel workers beat 1? For a job that just computes, will they?

import time, threading, multiprocessing as mp

def io_task():   time.sleep(1)                 # WAITING — pure I/O
def cpu_task():  sum(i*i for i in range(20_000_000))   # COMPUTING — pure CPU

def run(fn, workers, Kind):
    t = time.time()
    ws = [Kind(target=fn) for _ in range(workers)]
    [w.start() for w in ws]; [w.join() for w in ws]
    return round(time.time() - t, 2)

# 4 WAITING tasks: threads (concurrency) nearly tie — the sleeps overlap
print('4x io  · 1 thread  :', run(io_task, 1, threading.Thread))
print('4x io  · 4 threads :', run(io_task, 4, threading.Thread))   # ~same as 1, ≈1s

# 4 COMPUTING tasks: threads DON'T help (GIL), processes DO (real parallelism)
print('4x cpu · 4 threads :', run(cpu_task, 4, threading.Thread))  # ~4x one task (no speedup)
print('4x cpu · 4 procs   :', run(cpu_task, 4, mp.Process))        # ~1x one task (4x faster)

You'll watch the decider prove itself: four waiting tasks finish in ~1 second whether you use 1 worker or 4 (concurrency overlapped the waits), while four computing tasks only speed up with processes, not threads — because threads gave you concurrency, and only processes gave you parallelism. The gap between your prediction and the stopwatch is the lesson.

The distinction that reorganizes everything:

• Concurrency = dealing with many things at once (structure; interleaving; you write it). Parallelism = doing many things at once (execution; simultaneous; the hardware provides it). One core can be concurrent (via time-slicing), never parallel.
• The decider: is the bottleneck waiting (I/O-bound → concurrency is enough, even better) or computing (CPU-bound → you need real parallelism)? Match the model to the bottleneck.
• Amdahl's ceiling: the serial fraction caps speedup at 1/f — 10% serial means 10× max, and only ~4.7× on 8 cores. More workers hit diminishing returns.
• It names the whole section: the event loop is concurrency, the GIL is concurrency without parallelism, threads-across-cores is parallelism, and a small pool matches the DB's real parallel capacity.
• Neither is free — don't add either until a measured bottleneck demands it.

And now the cliffhanger this section has been walking toward. Parallelism means multiple workers executing at the same instant — and the moment two of them read and write the same piece of memory at that same instant, something quietly breaks. A counter that should say 100 says 97. A balance goes negative. This is the dark side of shared state, and it's why one of the oldest tools in computing exists. Next: Shared State & Race Conditions — Why Locks Exist.