From Numbers to Architecture

Take one feature — say, "users can post comments" — and build it twice: once for a product doing 100 requests per second, once for one doing 100,000. The code is nearly identical. The systems around that code share almost nothing: one is a single box you could run under a desk; the other is sharded databases, queues, cache fleets, and a team that thinks about geography.

This is the lesson the whole section has been building toward. You can run the ladder, you know the numbers, you've done the reps — but an estimate only matters if it lands on a design, and this is the landing: a map from the number you estimated to the architecture it implies. Five bands, one per factor of ten, each with its own correct shape — and, just as important, its own correct restraint, because building band four for band one traffic is as real a failure as the opposite.

By the end you'll read any QPS figure the way a staff engineer does: not as a quantity, but as an address.

Architecture doesn't change smoothly with load — it changes in bands, roughly one per 10×. Here is the map, then each band in the words you'd say at a whiteboard:

band	QPS	the shape	the next ceiling
1	1–100	one box: monolith + DB	availability
2	100–1K	LB + 2–3 app servers + DB + cache	DB reads
3	1K–10K	fleet + cache + replicas + CDN	writes & working set
4	10K–100K	+ queues + sharding + specialized stores	geography, blast radius
5	100K+	multi-region, cells	(a team owns this)

Band 1 — the one-box world (1–100 QPS). A monolith and Postgres, possibly on the same machine, no cache, no load balancer. This isn't the embarrassing phase before real architecture — it is the correct architecture for the traffic. A hundred QPS against a box that can do thousands leaves you a 10× cushion for free. The winning moves are boring: proven tech, backups, ship features. Most products live and die in this band — and the deaths are never from load.

Band 2 — a few boxes (100–1K QPS). The first real transition, and note why it happens: not capacity — availability. One box that dies takes you offline; so a load balancer and a second stateless app server appear (the reason state had to move out of app servers back in Stateful vs Stateless). A cache slides in front of the database because the read ratio makes it nearly free. You're still one database. Peak thinking starts mattering here: at ×5, a "400 QPS" service brushes the next band every lunchtime.

Band 3 — the working fleet (1K–10K QPS). Autoscaled app servers behind a real load balancer, cache mandatory rather than optional, read replicas taking the read flood off the primary, a CDN for anything static, and observability you actually look at. Hold this picture in your head, because most successful products live in this band permanently — and one of the most instructive systems on the internet proves it, as you'll see two sections down.

Band 4 — serious scale (10K–100K QPS). Now the hard wall: writes. Replicas multiplied your read capacity but every write still lands on one primary, and a single node has a real ceiling (the number sheet plans ~5–10K mixed QPS). So queues appear to absorb write bursts, the sharding conversation starts — carefully, because sharding is a one-way door: cross-shard queries and transactions get expensive forever, which is why the canon says exhaust caching, replicas, and vertical scaling first — and data splits into specialized stores. Someone's job title now contains the word infrastructure.

Band 5 — planet scale (100K+ QPS). Multiple regions, cell-based isolation, custom infrastructure, and the problems change character entirely — tail latency, fanout, blast radius. This course's third container lives here. You will not arrive by surprise; there will be a team.

Two things about the map deserve a why.

Why bands at all? Because each transition is a saturation event, not a gradient. Nothing about your architecture is wrong at 2× the load — the same shape just runs warmer (recall the utilization knee: warmer has a price, but the shape holds). At ~10×, some specific number crosses some specific ceiling — a box's cores, a primary's write throughput, a NIC — and the shape must change. Growth feels continuous; architecture responds in steps.

Why ~10× per band? Because that's roughly how far each shape's headroom stretches. A well-run single box genuinely covers two orders of magnitude; a fleet-with-replicas covers about one more; and each new shape buys the next factor of ten. This is also why the estimation habit of thinking in orders of magnitude — one digit and an exponent — turns out to be exactly the right resolution: an estimate's job is to place you in a band, and bands are an order of magnitude wide. A 2× estimation error almost never changes the band; a 10× error always does. That's the precision contract the whole section has been operating under, now justified.

"Architecture changes when a number crosses a ceiling." Name the ceilings and you can predict your next transition instead of discovering it at 2 a.m.:

• The single-primary write ceiling (~5–10K mixed QPS as a planning number). Replicas multiply reads; nothing multiplies writes except sharding or queues. This one ceiling drives the band 3 → 4 transition almost everywhere.
• Working set vs RAM. The moment your hot data outgrows memory — the box's or the cache tier's — latency changes species (100 ns vs 150 µs, the ratio you memorized). Note it's the hot slice that matters, not total data: Zipf means the hot 1% carries most traffic.
• The NIC. 10 Gbps is 1.25 GB/s; anything media-heavy hits the wire before it hits the CPU (the walkthroughs' video service found this ceiling from the other side).
• Connection counts. Databases are happy with hundreds of connections, not tens of thousands — which is why connection pooling quietly appears in every band 3 diagram.
• Geography. 150 ms across an ocean is frozen physics. Once your latency budget can't afford the trip, no amount of fleet fixes it — only regions and CDNs do. This ceiling triggers on where users are, not on QPS, which is why some 500-QPS products go multi-region before some 50K-QPS ones.

Storage has its own quiet bands, forcing transitions on a different axis: gigabytes live happily inside the database; terabytes demand dedicated thinking (replica sizing, backup windows measured in hours); petabytes mean object storage with lifecycle tiers while the database keeps only pointers — exactly the split the photo-app walkthrough landed on.

Under-building fails loudly — pages go down, everyone learns the lesson. Over-building fails silently: months of engineering spent on a shape the traffic never required. The evidence for restraint is embarrassingly strong:

• Stack Overflow serves ~2 billion pageviews a month — about 6,000 requests per second, pages rendered in ~12 ms — on nine web servers and one SQL Server (plus a hot standby, with 1.5 TB of RAM doing the cache-in-memory trick from the number sheet), two Redis boxes, a monolith, on-prem. Band 3 numbers, a disciplined band 3 architecture, world-class results.
• WhatsApp ran ~900M users with ~50 engineers — planet-scale traffic, one-box-anchor thinking.
• Amazon Prime Video publicly moved a workload from microservices back to a monolith and cut its cost ~90%. Segment consolidated 140+ services for the same reason. Kelsey Hightower's line — "monoliths are the future" — is only half a joke.

The failure mechanism is always the same: a team pays the distributed tax — network calls where function calls used to be, distributed debugging, a fallacy list's worth of new failure modes — at 200 QPS, where it buys nothing. The tax is only worth paying when a ceiling actually forces it.

So the rule, worth saying exactly: build for now, design for 10×, sketch 100×. Ten-x is one band up — the horizon you can genuinely see, and designing for it means knowing your exits (which ceiling, which lever) rather than pre-building them. A thousand-x is a different company; designing for it today is fiction with a budget.

Everything in this section compresses into a ritual you can run on any design — yours or the one you're reviewing — in under two minutes:

1. Which band are we in? Run the envelope (users → actions → ÷10⁵ → ×peak). One digit and an exponent places you on the map.
2. Which ceiling do we hit next? Name the specific number: the primary's writes? the working set vs RAM? the NIC? the ocean? If nobody can name it, that's the finding.
3. What's the 10× plan? The correct answer is often "nothing yet — noted": the plan is knowing which lever you'd pull (add replicas / add a queue / shard on key X), not pulling it early. A named exit costs a sentence; a pre-built one costs a quarter.

Ask these three of the comment-service example from the top of the lesson and the two worlds separate cleanly: at 100 QPS — band 1, next ceiling availability, plan "LB + second server when uptime matters"; at 100K — band 4→5, next ceiling geography, plan already a roadmap. Same feature. Different addresses.

A few traps to keep out of your own reviews: sizing for aspiration (designing the 1000× fantasy instead of the 10× horizon); band-skipping because a technology is fashionable (sharding a database that fits in RAM); forgetting ×5 peak and living quietly one band above your average; and mistaking total data for working set — the map runs on hot bytes and real requests, not on ambitions.

• A QPS figure is an address, not a quantity. Five bands, one per 10×: one box → a few boxes → the working fleet → queues-and-shards → planet scale. Each is correct for its traffic — including band 1.
• Transitions are saturations: a named number crossing a named ceiling — the write primary, the working set, the NIC, the ocean. Predict them; don't discover them.
• Estimates and bands share a resolution. Order-of-magnitude estimation is exactly precise enough to place you in a band — that's the contract that makes ninety-second envelopes worth doing.
• Over-building is failure too — Stack Overflow runs the internet's Q&A on nine web servers and a monolith. Build for now, design for 10× (know your exits), sketch 100×.
• The ritual: which band · which ceiling · what's the 10× plan. Two minutes, any design review.

That closes the section's arc: measure (percentiles), estimate (the ladder), know the numbers, walk the calculations, drill them, and now land them on architecture. The checkpoint ahead tests the whole chain — and then the Capstone hands you a blank marketplace and asks you to run every step of it for real.